-- Logs begin at Fri 2022-09-09 12:36:31 UTC, end at Fri 2022-09-09 13:36:36 UTC. -- Sep 09 13:00:13.005545 dv-cinder-831247-26-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 13:00:13.014945 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 13:00:13 2022] *** Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: nodename: dv-cinder-831247-26-iscsi Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: machine: x86_64 Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: clock source: unix Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: pcre jit disabled Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: detected number of CPU cores: 4 Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: current working directory: / Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: detected binary path: /usr/bin/uwsgi-core Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: your processes number limit is 47680 Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: your memory page size is 4096 bytes Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: detected max file descriptor number: 2048 Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: lock engine: pthread robust mutexes Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: thunder lock: enabled Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 13:00:13.019143 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 13:00:13.058360 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: Python main interpreter initialized at 0x5587c96969c0 Sep 09 13:00:13.058360 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: python threads support enabled Sep 09 13:00:13.058360 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: your server socket listen backlog is limited to 100 connections Sep 09 13:00:13.058360 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: your mercy for graceful operations on workers is 90 seconds Sep 09 13:00:13.058766 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 13:00:13.058766 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: *** Operational MODE: preforking *** Sep 09 13:00:13.059037 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: *** uWSGI is running in multiple interpreter mode *** Sep 09 13:00:13.059037 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: spawned uWSGI master process (pid: 81586) Sep 09 13:00:13.059259 dv-cinder-831247-26-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 13:00:13.060478 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: spawned uWSGI worker 1 (pid: 81602, cores: 1) Sep 09 13:00:13.060478 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: spawned uWSGI worker 2 (pid: 81603, cores: 1) Sep 09 13:00:13.061261 dv-cinder-831247-26-iscsi devstack@keystone.service[81586]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 13:00:17.557413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 13:00:17.557413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: from cryptography.utils import int_from_bytes Sep 09 13:00:17.558214 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 13:00:17.558214 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: from cryptography.utils import int_from_bytes Sep 09 13:00:17.558337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 13:00:17.558337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: from cryptography.utils import int_from_bytes Sep 09 13:00:17.558337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 13:00:17.558337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: from cryptography.utils import int_from_bytes Sep 09 13:00:17.608896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 13:00:17.609073 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 13:00:17.609161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 13:00:17.609241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 13:00:17.609304 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] command line args: [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 13:00:17.609474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] command line args: [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 13:00:17.609571 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 13:00:17.609742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ================================================================================ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 13:00:17.610342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 13:00:17.610342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ================================================================================ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 13:00:17.610486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] admin_token = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.610571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] admin_token = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.610644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] config_dir = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.610644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.610785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config_dir = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.610858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] config_source = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.610934 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.611012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] debug = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.611101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config_source = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.611101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] debug = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.611221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.611562 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.611661 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.611724 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.611787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.611847 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.611910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.611980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.612039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.612096 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.612153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.612215 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.612279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_config_append = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.612344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_config_append = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.612408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.612699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.612785 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_dir = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.612845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_dir = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.612902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_file = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.612972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.613031 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.613088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.613201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.613343 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.613432 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.613545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.613684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.613800 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.613876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.613958 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.614042 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.614124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.614255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.614451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.614629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.614815 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.615002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.615187 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.615360 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.615532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.615762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.616522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.616734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.616894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.617080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.617261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.617445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.617616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.617616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.617992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.618166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.618166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] publish_errors = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.618508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.618629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.618775 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] publish_errors = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.618899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.619027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.619197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.619322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.619445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.619602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.619738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.619893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.620013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] standard_threads = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.620128 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.620243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.620370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] standard_threads = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.620488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.620604 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.620719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.620840 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.620957 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_journal = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.621136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.621288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_json = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.621404 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_journal = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.621520 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_stderr = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.621642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_json = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.621759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_syslog = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.621883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_stderr = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.622007 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.622125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_syslog = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.622125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 13:00:17.622348 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.622348 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.622578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.622578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.622809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.622932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.623074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.623203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.623338 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.623457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.623575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.623696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.623846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.623994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.624123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.624240 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.624363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.624479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.624602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.624725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.624850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.624967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.625083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.625206 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.625327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.625443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.625569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.625695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.625811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.625927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.626043 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.626168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.626295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.626412 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.626529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.626645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.626767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.626884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.627087 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.627208 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.627324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.627439 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.627555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.627683 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.627841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.627962 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.628078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.628199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.628316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.628439 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.628558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.628674 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.628791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.628906 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.629020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.629143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.629266 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.629390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.629506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.629621 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.629736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.629862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.629999 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.630155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.630273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.630389 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.630509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.630637 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.630761 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.630879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.630995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.631135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.631251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.631379 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.631497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.631619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.631741 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.631857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.631974 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.632144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.632304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.632423 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.632539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.632655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.632769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.632965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.633093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.633210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.633210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.633434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.633550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.633674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.633791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.633906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.634023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.634145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.634145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.634370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.634487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.634602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.634708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.634807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.634910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.635017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.635144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.635256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.635365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.635466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.635571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.635684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.635804 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.635917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.636068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.636198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.636324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.636450 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.636566 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.636677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.636792 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.636907 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.637023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.637139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.637254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.637370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.637485 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.637610 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.637738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.637867 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.638039 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.638148 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.638271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.638388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.638506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.638628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.638754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.638872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.638989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.639120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.639250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.639369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.639494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.639602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.639717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.639839 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.639956 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.640082 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.640198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.640315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.640432 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.640548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.640664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.640780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.640905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.641023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.641140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.641255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.641373 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.641489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.641618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.641736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.641853 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.641970 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.642086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.642203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.642328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.642447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.642564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.642680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.642797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.642917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.643043 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.643194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.643311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.643428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.643557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.643683 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.643810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.643927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.644051 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.644169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.644292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.644409 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.644598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.644718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.644840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.644958 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.645075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.645201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.645328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.645458 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.645578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.645695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.645811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.645928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.646045 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.646170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.646170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.646390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.646506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.646624 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.646740 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.646865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.647506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.647624 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.647767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.647885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.648045 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.648175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.648300 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.648418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.648535 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.648651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.648778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.648926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.649054 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.external = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.649181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.external = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.649300 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.649418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.649535 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.649662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.649783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.649909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.650034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.password = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.650152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.password = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.650269 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.token = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.650386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.token = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.650510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.650636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.650745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.650869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.650869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.651148 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.651269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.651399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.651519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.651636 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.651755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.651873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.651990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.652115 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.652234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.652358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.652476 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.652602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.652721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.652852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.652974 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.653092 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.653209 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.653327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.653444 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.653570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.653688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.653806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.653930 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.654049 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.654167 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.654292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.654411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.654529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.654647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.654764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.654881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.655009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.655139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.655258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.655376 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.655493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.655611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.655741 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.655867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.655985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.656104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.656221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.656338 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.656466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.656586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.656703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.656820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.656938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.657054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.657183 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.657302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.657419 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.657538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.657654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.657780 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.657899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.658017 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.658176 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.658294 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.658412 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.658539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.658666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.658784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.658901 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.659018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.659188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.659385 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.659508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.659626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.659744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.659862 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.659980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.660107 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.660227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.660346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.660463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.660581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.660697 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.660831 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.660956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.661074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.661192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.661311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.661428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.661554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.661673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.661673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.661893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.662017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.662135 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.662261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.662388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.662506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.662662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.663272 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.663390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.663548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.663673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.663791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.663916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.664035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.664153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.664279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.664398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.664517 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.664634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.664752 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.664870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.664996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.665115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.665232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.665349 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.665468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.665585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.665711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.665830 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.665948 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.666066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.666183 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.666307 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.666434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.666554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.666672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.666789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.666907 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.667024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.667174 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.667295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.667295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.667530 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.667699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.667829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.667947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.668064 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.668213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.668333 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.668450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.668576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.668696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.668813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.668930 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.669048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.669166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.669292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.669410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.669528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.669528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.669748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.669865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.669990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.669990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.670211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.670329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.670445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.670562 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.670687 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.670807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.670924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.671041 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.671181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.671309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.671437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.671557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.671675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.671793 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.671910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.672027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.672152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.672271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.672388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.672388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.672609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.672731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.672858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.672978 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.673096 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.673213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.673331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.673448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.673573 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.673693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.673817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.673936 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.674053 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.674170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.674295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.674415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.674533 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.674649 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.674767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.674884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.675011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.675147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.675265 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.675383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.675500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.675617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.675747 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.675867 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.675986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.676103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.676221 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.676338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.676467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.676586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.676704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.676822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.676939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.677058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.677187 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.677306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.677414 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.677514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.677613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.677730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.677856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.677975 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.678094 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.678249 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.678370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.678487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.678613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.678732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.678849 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.678967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.679126 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.679246 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.679376 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.679496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.679613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.679730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.679848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.679965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.680091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.680211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.680329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.680447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.680564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.680681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.680806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.680925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.681043 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.681160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.681278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.681396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.681522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.681641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.681758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.681876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.681993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.682110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.682236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.682354 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.682472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.682592 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.682711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.682899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.683019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.683019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.683249 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.683368 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.683486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] role.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.683616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.683736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] role.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.683855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.683978 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.684097 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.684214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.684341 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.684461 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.684579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.684696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.684813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.684931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.685057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.685176 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.685294 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.685413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.685530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.685647 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.685774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.685893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.686011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.686128 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.686245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.686362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.686489 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.686608 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.686727 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.686844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.686962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.687123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.687256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.687376 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.687496 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.687613 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.687730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.687848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.687974 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.688093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.688211 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.688359 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.688480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.688611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.688731 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.688849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.688967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.689085 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.689202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.689328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.689447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.689565 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.689684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.689801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.689918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.690044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.690163 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.690281 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.690399 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.690516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.690634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.690759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.690878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.690996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.691128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.691246 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.691364 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.691493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.691613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.691732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.691849 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.691966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.692084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.692209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.692327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.692444 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.692565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.692682 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.692682 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.692912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.693031 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.693148 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.693272 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.693390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.693507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.693633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.693752 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.693870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.693987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.694104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.694222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.694347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.694464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.694582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.694699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.694818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.694935 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.695112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.695234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.695352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.695469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.695600 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.695732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.695860 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.695980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.696098 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.696224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.696333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.696449 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.696566 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.696698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.696817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.696944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.697062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.697180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.697296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.697413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.697530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.697656 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.697774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.697892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.698010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.698127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.698245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.698411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.698533 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.698651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.698768 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.698768 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.698988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.699134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.699255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.699374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.699491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.699609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.699726 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.699853 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.699972 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.700090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.700208 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.700325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.700443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.700568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.700687 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 13:00:17.700804 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 13:00:17.700922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 13:00:17.726680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.726908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.726998 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.727158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.727244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.727381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.735601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.735701 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.763675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.763911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.764107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.764337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.818176 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.818421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.868252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.868491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.869543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:17.869779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.025341 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.025575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.025791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.025876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.026014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.026101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.026234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.026319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.026455 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.026549 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.026716 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.026930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.049434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.049624 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.049717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.049883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.049970 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.050111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.050196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.050333 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.050426 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.050551 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.050635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.050765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.050848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.050980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.062036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.062480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.075192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.075382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.075610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.075706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.075832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.075916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.076050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.076134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.076265 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.076351 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.076482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.076566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.076732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.076944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 13:00:18.093532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.093747 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 13:00:18.094006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.094089 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.094216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 13:00:18.094315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 13:00:18.094550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.094634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.094747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 13:00:18.094830 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.095020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.095125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.095312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.095395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.095573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.095831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.105744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.106033 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.106311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.106394 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.106592 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.106688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.106864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.107184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.113648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.114123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.116693 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.116980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.117267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.117537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.117621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.117823 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.117920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.118088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.118170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.118363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.118448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.118642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.118726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.118902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.118985 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.119197 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.119295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.119472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.119554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.119738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.120010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.120277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.142769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.143640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.144281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.144574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.145102 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.145388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.149353 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.149647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.149934 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.150375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.150664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.150950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.156337 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.156630 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.157264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.157550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.162270 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.163247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.163550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.163846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.164133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.164231 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.164421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.164506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.164702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.164786 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.164979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.165064 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.165255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.165534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.165806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.166075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.180760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.181044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.182190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.182471 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.184960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.186331 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.188253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.188556 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.189582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.189880 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.192821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.194806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.196802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.198050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.200039 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.200347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.200635 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.200921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.201202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.201286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.201487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.201573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.201857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.202141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.202422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.202701 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.217677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.217972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.218245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.218515 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.219081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.219362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.219634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.219899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.225965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.227536 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.227663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.229060 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.229147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.229344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.230634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.230911 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.232029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.232323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.232611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.232900 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.233301 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.233571 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.233858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.234142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.234426 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.234826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.244671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.244963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.245250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.245536 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.245924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.246211 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.246428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.246512 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.246715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.246998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.247306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.247687 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.247967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.248253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.262743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.264446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.265702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.267369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.268650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.270144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.270275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.271666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.271817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.272510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.272931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.274029 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.274314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.274714 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.279455 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.280926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.281081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.282548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.283879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.284286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.284580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.284866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.285449 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.285844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.286132 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.286406 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.292816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.293114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.293416 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.294569 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.294862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.295178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.300358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.300650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.300941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.301227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.301521 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.301804 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.302009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 13:00:18.302102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.302295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.302380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.302578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.302661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.302853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.302946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.303236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.303336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.303525 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.303764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.303872 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.304096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.304203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.304380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.304655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.304938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.305231 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.305496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.305796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.306139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 13:00:18.339814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5587c96969c0 pid: 81602 (default app) Sep 09 13:00:18.341831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5587c96969c0 pid: 81603 (default app) Sep 09 13:00:18.343031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df85c60-c12c-4c04-b2a5-544f93a44ff3 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:18.343341 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df85c60-c12c-4c04-b2a5-544f93a44ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:18.343619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df85c60-c12c-4c04-b2a5-544f93a44ff3 None None] PATH_INFO: `/v3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:18.347775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Fri Sep 9 13:00:18 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:25.138821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b384e72-b2ea-410c-93c5-f0926cef6531 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:25.139364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b384e72-b2ea-410c-93c5-f0926cef6531 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:25.139470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b384e72-b2ea-410c-93c5-f0926cef6531 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:25.143725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:00:25.149581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1ab13-4081-4c5c-aabb-ae1e922dbf51 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:25.149985 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1ab13-4081-4c5c-aabb-ae1e922dbf51 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:25.150121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1ab13-4081-4c5c-aabb-ae1e922dbf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:25.220859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: INFO dbcounter [None req-35f1ab13-4081-4c5c-aabb-ae1e922dbf51 None None] Registered counter for database keystone Sep 09 13:00:25.240145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG oslo_db.sqlalchemy.engines [None req-35f1ab13-4081-4c5c-aabb-ae1e922dbf51 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81602) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 13:00:25.246476 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writer thread running {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 13:00:25.425273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG passlib.handlers.bcrypt [None req-35f1ab13-4081-4c5c-aabb-ae1e922dbf51 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81602) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 13:00:25.425664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG passlib.handlers.bcrypt [None req-35f1ab13-4081-4c5c-aabb-ae1e922dbf51 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81602) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 13:00:25.483909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-35f1ab13-4081-4c5c-aabb-ae1e922dbf51 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:25.485528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-35f1ab13-4081-4c5c-aabb-ae1e922dbf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:25.615240 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 13:00:26.081422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b384e72-b2ea-410c-93c5-f0926cef6531 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:26.151197 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: INFO dbcounter [None req-7b384e72-b2ea-410c-93c5-f0926cef6531 None None] Registered counter for database keystone Sep 09 13:00:26.172642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG oslo_db.sqlalchemy.engines [None req-7b384e72-b2ea-410c-93c5-f0926cef6531 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 13:00:26.179332 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writer thread running {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 13:00:26.297769 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b384e72-b2ea-410c-93c5-f0926cef6531 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:26.302068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece097e-034c-4f01-b37a-d3671afe90b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:26.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece097e-034c-4f01-b37a-d3671afe90b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:26.302963 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece097e-034c-4f01-b37a-d3671afe90b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:26.303263 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece097e-034c-4f01-b37a-d3671afe90b8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:26.311400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece097e-034c-4f01-b37a-d3671afe90b8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:26.362933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3ece097e-034c-4f01-b37a-d3671afe90b8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 13:00:26.363228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece097e-034c-4f01-b37a-d3671afe90b8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:26.372499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Fri Sep 9 13:00:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:00:26.378435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1ab13-4081-4c5c-aabb-ae1e922dbf51 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:26.396376 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1ab13-4081-4c5c-aabb-ae1e922dbf51 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:26.400782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7ccc3-c09a-4054-9bde-21dcf96d4e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:26.401898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7ccc3-c09a-4054-9bde-21dcf96d4e8b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:26.402188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7ccc3-c09a-4054-9bde-21dcf96d4e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:26.402468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7ccc3-c09a-4054-9bde-21dcf96d4e8b admin admin] PATH_INFO: `/v3/projects/97408104e346493ebafc4d549d2257c4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:26.404093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7ccc3-c09a-4054-9bde-21dcf96d4e8b admin admin] RBAC: Authorizing `identity:get_project(project_id=97408104e346493ebafc4d549d2257c4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:26.456497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: WARNING keystone.common.rbac_enforcer.enforcer [None req-64a7ccc3-c09a-4054-9bde-21dcf96d4e8b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 13:00:26.456845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7ccc3-c09a-4054-9bde-21dcf96d4e8b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:26.461886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Fri Sep 9 13:00:26 2022] GET /identity/v3/projects/97408104e346493ebafc4d549d2257c4 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:28.578491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee557d-db3d-45c4-bb36-b3becdbcf8b5 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:28.579025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee557d-db3d-45c4-bb36-b3becdbcf8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:28.579025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee557d-db3d-45c4-bb36-b3becdbcf8b5 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:28.583546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:28.589197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258297a1-1fb6-4f92-985a-3f754a898df8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:28.589602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258297a1-1fb6-4f92-985a-3f754a898df8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:28.589756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258297a1-1fb6-4f92-985a-3f754a898df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:28.610208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-258297a1-1fb6-4f92-985a-3f754a898df8 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:28.611775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-258297a1-1fb6-4f92-985a-3f754a898df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:28.620159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 13:00:28.885212 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee557d-db3d-45c4-bb36-b3becdbcf8b5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:28.897325 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee557d-db3d-45c4-bb36-b3becdbcf8b5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:28.901502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839201b-dffa-4425-973d-de1aa3b0739d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:28.902100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839201b-dffa-4425-973d-de1aa3b0739d admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:28.902378 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839201b-dffa-4425-973d-de1aa3b0739d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:28.902648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839201b-dffa-4425-973d-de1aa3b0739d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:28.910131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839201b-dffa-4425-973d-de1aa3b0739d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:28.911924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839201b-dffa-4425-973d-de1aa3b0739d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:28.920485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Fri Sep 9 13:00:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:00:28.926227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258297a1-1fb6-4f92-985a-3f754a898df8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:28.935323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258297a1-1fb6-4f92-985a-3f754a898df8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:28.939658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a27875-260a-4561-9225-f8072c39b54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:28.940282 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a27875-260a-4561-9225-f8072c39b54f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:28.940561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a27875-260a-4561-9225-f8072c39b54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:28.940856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a27875-260a-4561-9225-f8072c39b54f admin admin] PATH_INFO: `/v3/users/a4f7da3ad6214b7b94cd5b55e583b917` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:28.942806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a27875-260a-4561-9225-f8072c39b54f admin admin] RBAC: Authorizing `identity:get_user(user_id=a4f7da3ad6214b7b94cd5b55e583b917)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:28.944654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a27875-260a-4561-9225-f8072c39b54f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:28.946206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Fri Sep 9 13:00:28 2022] GET /identity/v3/users/a4f7da3ad6214b7b94cd5b55e583b917 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:00:31.164253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24522110-13fb-45f3-883e-8d74429b620c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:31.164773 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24522110-13fb-45f3-883e-8d74429b620c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:31.164882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24522110-13fb-45f3-883e-8d74429b620c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:31.166140 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:00:31.171513 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fcb22-68a9-452d-88ce-9667fad05dda None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:31.171929 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fcb22-68a9-452d-88ce-9667fad05dda None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:31.172058 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fcb22-68a9-452d-88ce-9667fad05dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:31.190588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f78fcb22-68a9-452d-88ce-9667fad05dda None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:31.192739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f78fcb22-68a9-452d-88ce-9667fad05dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:31.198728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70509624-3cf9-4b6d-b1bb-c91d3dc242f8 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:31.199010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70509624-3cf9-4b6d-b1bb-c91d3dc242f8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:31.199327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70509624-3cf9-4b6d-b1bb-c91d3dc242f8 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:31.200635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:00:31.206104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b1d46-ce8c-4a5c-befc-37571c4a7dbe None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:31.206392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b1d46-ce8c-4a5c-befc-37571c4a7dbe None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:31.206669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b1d46-ce8c-4a5c-befc-37571c4a7dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:31.248319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG passlib.handlers.bcrypt [None req-7b0b1d46-ce8c-4a5c-befc-37571c4a7dbe None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81603) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 13:00:31.248716 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG passlib.handlers.bcrypt [None req-7b0b1d46-ce8c-4a5c-befc-37571c4a7dbe None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81603) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 13:00:31.274696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:31.298196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7b0b1d46-ce8c-4a5c-befc-37571c4a7dbe None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:31.299830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7b0b1d46-ce8c-4a5c-befc-37571c4a7dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:31.402511 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:00:31.545900 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fcb22-68a9-452d-88ce-9667fad05dda None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:31.570253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:31.653207 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fcb22-68a9-452d-88ce-9667fad05dda None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:31.671778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b1d46-ce8c-4a5c-befc-37571c4a7dbe None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:31.686962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce4829-e898-4d92-9b59-9168bad0dd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:31.687614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4829-e898-4d92-9b59-9168bad0dd27 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:31.687891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4829-e898-4d92-9b59-9168bad0dd27 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:31.688162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4829-e898-4d92-9b59-9168bad0dd27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:31.700182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce4829-e898-4d92-9b59-9168bad0dd27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-02ce4829-e898-4d92-9b59-9168bad0dd27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:31.709778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:31.710916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:31.711993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:31.711993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:31.711993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:31.711993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:31.711993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:31.711993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:31.711993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:31.711993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:31.711993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:31.711993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:31.711993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:00:31.711993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:00:31.711993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:00:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:31.718571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce4829-e898-4d92-9b59-9168bad0dd27 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:31.775030 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b1d46-ce8c-4a5c-befc-37571c4a7dbe None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:31.777347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:31.800771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:31.812296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce1104-35dd-49a2-8f2e-7482a819633c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:31.813141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce1104-35dd-49a2-8f2e-7482a819633c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:31.813433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce1104-35dd-49a2-8f2e-7482a819633c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:31.813704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce1104-35dd-49a2-8f2e-7482a819633c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:31.819651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce4829-e898-4d92-9b59-9168bad0dd27 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:31.853239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab555d-6f55-4bd8-bbc8-37424ae7dae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:31.853844 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab555d-6f55-4bd8-bbc8-37424ae7dae4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:31.854119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab555d-6f55-4bd8-bbc8-37424ae7dae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:31.854391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab555d-6f55-4bd8-bbc8-37424ae7dae4 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:31.855066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab555d-6f55-4bd8-bbc8-37424ae7dae4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:31.855384 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab555d-6f55-4bd8-bbc8-37424ae7dae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:31.856774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab555d-6f55-4bd8-bbc8-37424ae7dae4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:31.868948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:00:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:00:31.874501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab555d-6f55-4bd8-bbc8-37424ae7dae4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:31.882757 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce1104-35dd-49a2-8f2e-7482a819633c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:31.884344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce1104-35dd-49a2-8f2e-7482a819633c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:31.974774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab555d-6f55-4bd8-bbc8-37424ae7dae4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:31.982937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:00:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 13:00:31.988974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce1104-35dd-49a2-8f2e-7482a819633c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:31.991797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:32.009102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8337c-9e95-4d19-9c90-59fcab026e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:32.009740 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8337c-9e95-4d19-9c90-59fcab026e4d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:32.010020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8337c-9e95-4d19-9c90-59fcab026e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:32.010292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8337c-9e95-4d19-9c90-59fcab026e4d None admin] PATH_INFO: `/v3/users/a4f7da3ad6214b7b94cd5b55e583b917` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:32.011468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8337c-9e95-4d19-9c90-59fcab026e4d None admin] RBAC: Authorizing `identity:get_user(user_id=a4f7da3ad6214b7b94cd5b55e583b917)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:32.012930 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8337c-9e95-4d19-9c90-59fcab026e4d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:32.014841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:31 2022] GET /identity/v3/users/a4f7da3ad6214b7b94cd5b55e583b917 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:00:32.020000 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8337c-9e95-4d19-9c90-59fcab026e4d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:32.033967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:32.089838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce1104-35dd-49a2-8f2e-7482a819633c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:32.118578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8337c-9e95-4d19-9c90-59fcab026e4d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:32.122842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fc90c-6470-4025-b490-f5c8995a4479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:32.123723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fc90c-6470-4025-b490-f5c8995a4479 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:32.124007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fc90c-6470-4025-b490-f5c8995a4479 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:32.124282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fc90c-6470-4025-b490-f5c8995a4479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:32.143858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fc90c-6470-4025-b490-f5c8995a4479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:32.145321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fc90c-6470-4025-b490-f5c8995a4479 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:32.153886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d317492-91ae-496b-a2ca-899bfbba528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:32.154523 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317492-91ae-496b-a2ca-899bfbba528d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:32.154805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317492-91ae-496b-a2ca-899bfbba528d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:32.155102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317492-91ae-496b-a2ca-899bfbba528d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:32.156213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d317492-91ae-496b-a2ca-899bfbba528d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:32.157734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d317492-91ae-496b-a2ca-899bfbba528d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:32.164139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 10/18] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:00:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-108fc90c-6470-4025-b490-f5c8995a4479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:00:32.164264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:32.166250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:00:32.167077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:00:32.167077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:00:32.167077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:00:32.167267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d317492-91ae-496b-a2ca-899bfbba528d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:32.167334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 9/19] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:00:32.170362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fc90c-6470-4025-b490-f5c8995a4479 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:32.228159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:32.249649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:32.269730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d317492-91ae-496b-a2ca-899bfbba528d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:32.274456 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fc90c-6470-4025-b490-f5c8995a4479 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:32.303693 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54411d-d962-4ea2-b48d-d807c19dd8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:32.304301 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54411d-d962-4ea2-b48d-d807c19dd8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:32.304579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54411d-d962-4ea2-b48d-d807c19dd8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:32.304852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54411d-d962-4ea2-b48d-d807c19dd8e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:32.305569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54411d-d962-4ea2-b48d-d807c19dd8e1 None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.domain.id=default, user.id=a4f7da3ad6214b7b94cd5b55e583b917) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:32.305875 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54411d-d962-4ea2-b48d-d807c19dd8e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:32.307339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54411d-d962-4ea2-b48d-d807c19dd8e1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:32.308335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738420c-bdf3-49f7-8ffd-4c5f120b3c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:32.308953 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738420c-bdf3-49f7-8ffd-4c5f120b3c6e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:32.309237 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738420c-bdf3-49f7-8ffd-4c5f120b3c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:32.309513 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738420c-bdf3-49f7-8ffd-4c5f120b3c6e None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:32.310155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738420c-bdf3-49f7-8ffd-4c5f120b3c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:32.310466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738420c-bdf3-49f7-8ffd-4c5f120b3c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:32.311879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738420c-bdf3-49f7-8ffd-4c5f120b3c6e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:32.313788 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 11/20] 10.222.0.17 () {56 vars in 1322 bytes} [Fri Sep 9 13:00:32 2022] GET /identity/v3/role_assignments?user.id=a4f7da3ad6214b7b94cd5b55e583b917&scope.domain.id=default&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:32.334950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 10/21] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:00:34.683629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e06e44-0387-4d9e-8a53-b9fc9568ae57 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:34.684583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e06e44-0387-4d9e-8a53-b9fc9568ae57 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:34.685061 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e06e44-0387-4d9e-8a53-b9fc9568ae57 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:34.686642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 12/22] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:34.693206 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c7495-f50a-447f-abd0-2646aaf0593c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:34.694083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c7495-f50a-447f-abd0-2646aaf0593c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:34.694571 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c7495-f50a-447f-abd0-2646aaf0593c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:34.715090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-632c7495-f50a-447f-abd0-2646aaf0593c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:34.716626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-632c7495-f50a-447f-abd0-2646aaf0593c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:34.808066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 11/23] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:34.989659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9af1b5-ebd4-4660-888d-c86a7b0e1e00 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:34.990334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9af1b5-ebd4-4660-888d-c86a7b0e1e00 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:34.990839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9af1b5-ebd4-4660-888d-c86a7b0e1e00 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:34.992641 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 13/24] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:35.001032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaaa55-9438-4040-b364-1befd8b6df74 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:35.001580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaaa55-9438-4040-b364-1befd8b6df74 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:35.002054 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaaa55-9438-4040-b364-1befd8b6df74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:35.023007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a7aaaa55-9438-4040-b364-1befd8b6df74 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:35.024228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a7aaaa55-9438-4040-b364-1befd8b6df74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:35.085162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:35.110580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 12/25] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:35.356521 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56418bef-5054-4847-91e2-790732e398ec None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:35.357157 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56418bef-5054-4847-91e2-790732e398ec None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:35.357642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56418bef-5054-4847-91e2-790732e398ec None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:35.359290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 14/26] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:35.365589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90d9ce-08eb-4872-b2cf-94524948401b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:35.366140 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90d9ce-08eb-4872-b2cf-94524948401b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:35.366618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90d9ce-08eb-4872-b2cf-94524948401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:35.380433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56418bef-5054-4847-91e2-790732e398ec None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:35.405066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-fb90d9ce-08eb-4872-b2cf-94524948401b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:35.407430 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fb90d9ce-08eb-4872-b2cf-94524948401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:35.505674 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 13/27] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:35.507350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b907c6-6303-43f2-9763-0f4e4ee4fc8e None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:35.507763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b907c6-6303-43f2-9763-0f4e4ee4fc8e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:35.508177 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b907c6-6303-43f2-9763-0f4e4ee4fc8e None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:35.509825 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 14/28] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:35.510655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b907c6-6303-43f2-9763-0f4e4ee4fc8e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:35.540243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56418bef-5054-4847-91e2-790732e398ec None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:35.617961 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b907c6-6303-43f2-9763-0f4e4ee4fc8e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:35.623402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bce51e-2795-4c64-9164-3ed88993a51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:35.624180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bce51e-2795-4c64-9164-3ed88993a51b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:35.624564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bce51e-2795-4c64-9164-3ed88993a51b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:35.624940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bce51e-2795-4c64-9164-3ed88993a51b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:35.626633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:35.651703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bce51e-2795-4c64-9164-3ed88993a51b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:35.653338 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bce51e-2795-4c64-9164-3ed88993a51b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:35.655818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364544e-d682-4e42-bfea-f809886e438b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:35.656696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364544e-d682-4e42-bfea-f809886e438b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:35.657184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364544e-d682-4e42-bfea-f809886e438b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:35.657284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:35.657748 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364544e-d682-4e42-bfea-f809886e438b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:35.670382 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364544e-d682-4e42-bfea-f809886e438b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8364544e-d682-4e42-bfea-f809886e438b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:35.677364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:35.678205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:35.678959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:35.678959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:35.678959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:35.678959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:35.678959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:35.678959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:35.678959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:35.678959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:35.678959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:35.678959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:35.678959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:00:35.678959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:00:35.681602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 15/29] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:00:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:35.682956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8390732-c235-47a9-bb1a-0c4fce0c4556 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-04bce51e-2795-4c64-9164-3ed88993a51b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:00:35.687151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:35.688288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:00:35.689085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:00:35.689085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:00:35.689085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:00:35.690021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8390732-c235-47a9-bb1a-0c4fce0c4556 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:35.690514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8390732-c235-47a9-bb1a-0c4fce0c4556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:35.691396 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 15/30] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:00:35.695964 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bce51e-2795-4c64-9164-3ed88993a51b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:35.713195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e8390732-c235-47a9-bb1a-0c4fce0c4556 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:35.714992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e8390732-c235-47a9-bb1a-0c4fce0c4556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:35.826084 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 16/31] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:35.826787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8390732-c235-47a9-bb1a-0c4fce0c4556 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:35.914819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bce51e-2795-4c64-9164-3ed88993a51b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:35.979211 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c7927-5fe2-4c08-bf05-f9105f936909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:35.979875 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c7927-5fe2-4c08-bf05-f9105f936909 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:35.980210 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c7927-5fe2-4c08-bf05-f9105f936909 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:35.980478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c7927-5fe2-4c08-bf05-f9105f936909 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:35.981179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c7927-5fe2-4c08-bf05-f9105f936909 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:35.981485 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c7927-5fe2-4c08-bf05-f9105f936909 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:35.982882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c7927-5fe2-4c08-bf05-f9105f936909 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:36.012795 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:36.018412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 16/32] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:36.018412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c7927-5fe2-4c08-bf05-f9105f936909 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:36.026334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8390732-c235-47a9-bb1a-0c4fce0c4556 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:36.076647 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ec5b3-2356-4874-9621-442fc8dafb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:36.077633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ec5b3-2356-4874-9621-442fc8dafb59 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:36.079280 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ec5b3-2356-4874-9621-442fc8dafb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:36.079784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ec5b3-2356-4874-9621-442fc8dafb59 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:36.082248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ec5b3-2356-4874-9621-442fc8dafb59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:36.082800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ec5b3-2356-4874-9621-442fc8dafb59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:36.084409 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ec5b3-2356-4874-9621-442fc8dafb59 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:36.112276 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 17/33] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:00:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:36.114221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ec5b3-2356-4874-9621-442fc8dafb59 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:36.135113 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:36.183741 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c7927-5fe2-4c08-bf05-f9105f936909 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:36.222958 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb69b5-e759-4bc1-bcc9-6c8ae8ebce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:36.223963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb69b5-e759-4bc1-bcc9-6c8ae8ebce1d None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:36.224415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb69b5-e759-4bc1-bcc9-6c8ae8ebce1d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:36.224855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb69b5-e759-4bc1-bcc9-6c8ae8ebce1d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:36.225692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb69b5-e759-4bc1-bcc9-6c8ae8ebce1d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:36.227471 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb69b5-e759-4bc1-bcc9-6c8ae8ebce1d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:36.250545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ec5b3-2356-4874-9621-442fc8dafb59 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:36.292388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1001af3-32de-4b88-aaa2-1092fcca053c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:36.293320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1001af3-32de-4b88-aaa2-1092fcca053c None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:36.293773 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1001af3-32de-4b88-aaa2-1092fcca053c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:36.294252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1001af3-32de-4b88-aaa2-1092fcca053c None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:36.295381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1001af3-32de-4b88-aaa2-1092fcca053c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:36.297056 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1001af3-32de-4b88-aaa2-1092fcca053c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:36.363481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 17/34] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 9 13:00:36 2022] POST /identity/v3/roles => generated 225 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:00:36.364963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb69b5-e759-4bc1-bcc9-6c8ae8ebce1d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:36.462257 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 18/35] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:00:36 2022] POST /identity/v3/projects => generated 301 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:00:36.472667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1001af3-32de-4b88-aaa2-1092fcca053c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:36.497484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:36.515229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb69b5-e759-4bc1-bcc9-6c8ae8ebce1d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:36.549966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:36.600634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc363afa-eefb-425e-84ca-0dd3db1a7eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:36.601315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc363afa-eefb-425e-84ca-0dd3db1a7eca None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:36.601603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc363afa-eefb-425e-84ca-0dd3db1a7eca None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:36.601881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc363afa-eefb-425e-84ca-0dd3db1a7eca None admin] PATH_INFO: `/v3/users/a4f7da3ad6214b7b94cd5b55e583b917` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:36.603289 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc363afa-eefb-425e-84ca-0dd3db1a7eca None admin] RBAC: Authorizing `identity:get_user(user_id=a4f7da3ad6214b7b94cd5b55e583b917)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:36.604927 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc363afa-eefb-425e-84ca-0dd3db1a7eca None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:36.606872 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 18/36] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:36 2022] GET /identity/v3/users/a4f7da3ad6214b7b94cd5b55e583b917 => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:36.608003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6da6b4-9292-4107-9cea-63de7498dd14 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:36.608293 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6da6b4-9292-4107-9cea-63de7498dd14 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:36.608581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6da6b4-9292-4107-9cea-63de7498dd14 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:36.610100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 19/37] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:36.620796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6da6b4-9292-4107-9cea-63de7498dd14 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:36.677398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1001af3-32de-4b88-aaa2-1092fcca053c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:36.722727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7246f3e-f973-4a59-9d8d-c339a8af0155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:36.722727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7246f3e-f973-4a59-9d8d-c339a8af0155 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:36.722727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7246f3e-f973-4a59-9d8d-c339a8af0155 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:36.722727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7246f3e-f973-4a59-9d8d-c339a8af0155 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:36.723517 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7246f3e-f973-4a59-9d8d-c339a8af0155 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:36.723767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7246f3e-f973-4a59-9d8d-c339a8af0155 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:36.745514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 19/38] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 9 13:00:36 2022] POST /identity/v3/roles => generated 219 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:00:36.747439 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf72e84-93da-4a57-8994-066246e8d065 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:36.747919 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf72e84-93da-4a57-8994-066246e8d065 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:36.748337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf72e84-93da-4a57-8994-066246e8d065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:36.779889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7bf72e84-93da-4a57-8994-066246e8d065 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:36.782015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7bf72e84-93da-4a57-8994-066246e8d065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:36.791519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6da6b4-9292-4107-9cea-63de7498dd14 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:36.842774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9362010-ba8e-4d23-876b-8472b19c662f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:36.843746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9362010-ba8e-4d23-876b-8472b19c662f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:36.844164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9362010-ba8e-4d23-876b-8472b19c662f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:36.844572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9362010-ba8e-4d23-876b-8472b19c662f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:36.846640 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9362010-ba8e-4d23-876b-8472b19c662f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:36.848774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9362010-ba8e-4d23-876b-8472b19c662f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:36.850986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 20/39] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:00:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:36.858254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9362010-ba8e-4d23-876b-8472b19c662f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:36.877640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:36.884090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 20/40] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:36.925265 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dd022-0090-4be6-9d5a-ac2beab690ef None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:36.925747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dd022-0090-4be6-9d5a-ac2beab690ef None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:36.926029 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dd022-0090-4be6-9d5a-ac2beab690ef None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:36.927739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 21/41] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:36.936763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a9b23-bab7-4763-ab04-4c904079db81 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:36.937041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a9b23-bab7-4763-ab04-4c904079db81 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:36.937322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a9b23-bab7-4763-ab04-4c904079db81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:36.946936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:36.967380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-476a9b23-bab7-4763-ab04-4c904079db81 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:36.968876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-476a9b23-bab7-4763-ab04-4c904079db81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:37.010054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9362010-ba8e-4d23-876b-8472b19c662f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:37.053625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6329fd-259e-4322-b188-3b26112b9639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:37.054303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6329fd-259e-4322-b188-3b26112b9639 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:37.054588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6329fd-259e-4322-b188-3b26112b9639 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:37.054872 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6329fd-259e-4322-b188-3b26112b9639 None admin] PATH_INFO: `/v3/domains/default/users/a4f7da3ad6214b7b94cd5b55e583b917/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:37.055976 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 22/42] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:00:37.056905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6329fd-259e-4322-b188-3b26112b9639 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a4f7da3ad6214b7b94cd5b55e583b917, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:37.058375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6329fd-259e-4322-b188-3b26112b9639 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:37.074219 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 21/43] 10.222.0.17 () {58 vars in 1361 bytes} [Fri Sep 9 13:00:36 2022] PUT /identity/v3/domains/default/users/a4f7da3ad6214b7b94cd5b55e583b917/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:00:37.167609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a9b23-bab7-4763-ab04-4c904079db81 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:37.277371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a9b23-bab7-4763-ab04-4c904079db81 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:37.306511 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:37.313104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a2255-61f5-4e91-b9aa-31dc7197dc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:37.313724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a2255-61f5-4e91-b9aa-31dc7197dc27 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:37.314007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a2255-61f5-4e91-b9aa-31dc7197dc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:37.314408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a2255-61f5-4e91-b9aa-31dc7197dc27 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:37.315724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a2255-61f5-4e91-b9aa-31dc7197dc27 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:37.317154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a2255-61f5-4e91-b9aa-31dc7197dc27 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:37.332072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6329fd-259e-4322-b188-3b26112b9639 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:37.333657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 23/44] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 9 13:00:37 2022] POST /identity/v3/roles => generated 223 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:00:37.448457 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6329fd-259e-4322-b188-3b26112b9639 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:37.462941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:37.490228 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2d73f-010a-4e7a-a960-c68e1463c49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:37.490886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2d73f-010a-4e7a-a960-c68e1463c49a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:37.491197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2d73f-010a-4e7a-a960-c68e1463c49a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:37.491477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2d73f-010a-4e7a-a960-c68e1463c49a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:37.492641 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2d73f-010a-4e7a-a960-c68e1463c49a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:37.494135 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2d73f-010a-4e7a-a960-c68e1463c49a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:37.496025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 22/45] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:00:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:00:37.501984 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a2255-61f5-4e91-b9aa-31dc7197dc27 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:37.611420 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a2255-61f5-4e91-b9aa-31dc7197dc27 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:37.649917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41f1be-f43c-4a34-8138-a1f2ebaa5b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:37.650566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41f1be-f43c-4a34-8138-a1f2ebaa5b43 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:37.650851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41f1be-f43c-4a34-8138-a1f2ebaa5b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:37.651156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41f1be-f43c-4a34-8138-a1f2ebaa5b43 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:37.651821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41f1be-f43c-4a34-8138-a1f2ebaa5b43 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:37.653280 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41f1be-f43c-4a34-8138-a1f2ebaa5b43 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:37.679777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 24/46] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:00:37 2022] POST /identity/v3/projects => generated 312 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:00:39.283397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33d479-f0af-485b-95ff-b947dfc15a0c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:39.283944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33d479-f0af-485b-95ff-b947dfc15a0c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:39.283944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33d479-f0af-485b-95ff-b947dfc15a0c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:39.285296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 23/47] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:39.290483 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85597a-4142-4391-82ac-0d9234311c88 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:39.290874 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85597a-4142-4391-82ac-0d9234311c88 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:39.291042 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85597a-4142-4391-82ac-0d9234311c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:39.308857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5f85597a-4142-4391-82ac-0d9234311c88 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:39.310043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5f85597a-4142-4391-82ac-0d9234311c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:39.345693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:39.378402 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 25/48] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:39.647234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33d479-f0af-485b-95ff-b947dfc15a0c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:39.848027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89643c-0e47-484a-ab83-fbe4febd0c16 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:39.848312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89643c-0e47-484a-ab83-fbe4febd0c16 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:39.848591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89643c-0e47-484a-ab83-fbe4febd0c16 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:39.849940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 26/49] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:00:39.854764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd135e5e-48a9-4867-931e-0ff73f3f7dd2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:39.855065 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd135e5e-48a9-4867-931e-0ff73f3f7dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:39.855348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd135e5e-48a9-4867-931e-0ff73f3f7dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:39.857794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33d479-f0af-485b-95ff-b947dfc15a0c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:39.874204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-cd135e5e-48a9-4867-931e-0ff73f3f7dd2 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:39.875377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cd135e5e-48a9-4867-931e-0ff73f3f7dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:39.892965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3761e-ee96-43a3-a313-2985f79efad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:39.893602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3761e-ee96-43a3-a313-2985f79efad0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:39.893889 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3761e-ee96-43a3-a313-2985f79efad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:39.894169 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3761e-ee96-43a3-a313-2985f79efad0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:39.904441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3761e-ee96-43a3-a313-2985f79efad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-17e3761e-ee96-43a3-a313-2985f79efad0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:39.909842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:39.910811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:39.912134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:39.912134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:39.912134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:39.912134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:39.912134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:39.912134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:39.912134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:39.912134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:39.912134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:39.912134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:39.912134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:00:39.912134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:00:39.912134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 24/50] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:00:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:39.917209 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3761e-ee96-43a3-a313-2985f79efad0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:39.966976 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:40.029661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3761e-ee96-43a3-a313-2985f79efad0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:40.049988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 27/51] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:00:40.063140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d361885-b026-4422-8242-cf6a55c06f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:40.064274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d361885-b026-4422-8242-cf6a55c06f88 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:40.064635 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d361885-b026-4422-8242-cf6a55c06f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:40.065059 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d361885-b026-4422-8242-cf6a55c06f88 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:40.065733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d361885-b026-4422-8242-cf6a55c06f88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:40.066044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d361885-b026-4422-8242-cf6a55c06f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:40.067469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d361885-b026-4422-8242-cf6a55c06f88 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:40.076127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 25/52] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:00:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:00:40.082599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd135e5e-48a9-4867-931e-0ff73f3f7dd2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:40.178804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd135e5e-48a9-4867-931e-0ff73f3f7dd2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:40.180374 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:40.213476 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2946440-1174-4420-8100-9d17bc9eb1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:40.214118 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2946440-1174-4420-8100-9d17bc9eb1ec None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:40.214403 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2946440-1174-4420-8100-9d17bc9eb1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:40.214681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2946440-1174-4420-8100-9d17bc9eb1ec None admin] PATH_INFO: `/v3/users/a4f7da3ad6214b7b94cd5b55e583b917` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:40.216106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2946440-1174-4420-8100-9d17bc9eb1ec None admin] RBAC: Authorizing `identity:get_user(user_id=a4f7da3ad6214b7b94cd5b55e583b917)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:40.217552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2946440-1174-4420-8100-9d17bc9eb1ec None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:40.219158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 28/53] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:40 2022] GET /identity/v3/users/a4f7da3ad6214b7b94cd5b55e583b917 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:00:40.225205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d361885-b026-4422-8242-cf6a55c06f88 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:40.316812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2946440-1174-4420-8100-9d17bc9eb1ec None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:40.323954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d361885-b026-4422-8242-cf6a55c06f88 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:40.325764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:40.359919 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8621d0-b090-456c-891c-c10f02f78ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:40.360627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8621d0-b090-456c-891c-c10f02f78ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:40.360966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8621d0-b090-456c-891c-c10f02f78ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:40.361246 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8621d0-b090-456c-891c-c10f02f78ce1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:40.362430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8621d0-b090-456c-891c-c10f02f78ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:40.363940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8621d0-b090-456c-891c-c10f02f78ce1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:40.365522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 26/54] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:00:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:40.371229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8621d0-b090-456c-891c-c10f02f78ce1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:40.419173 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2946440-1174-4420-8100-9d17bc9eb1ec None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:40.452643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8aedf4-a3e7-4736-927a-b3d383776810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:40.453106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8aedf4-a3e7-4736-927a-b3d383776810 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:40.453391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8aedf4-a3e7-4736-927a-b3d383776810 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:40.453659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8aedf4-a3e7-4736-927a-b3d383776810 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:40.455396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8aedf4-a3e7-4736-927a-b3d383776810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:40.456845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8aedf4-a3e7-4736-927a-b3d383776810 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:40.458410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 29/55] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:00:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:00:40.463829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8aedf4-a3e7-4736-927a-b3d383776810 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:40.468781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8621d0-b090-456c-891c-c10f02f78ce1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:40.503156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340568e-a0dd-45e4-9674-de6ffd480f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:40.504040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340568e-a0dd-45e4-9674-de6ffd480f4f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:40.504153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:40.504274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340568e-a0dd-45e4-9674-de6ffd480f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:40.504274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340568e-a0dd-45e4-9674-de6ffd480f4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:40.505085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340568e-a0dd-45e4-9674-de6ffd480f4f None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.domain.id=default, user.id=a4f7da3ad6214b7b94cd5b55e583b917) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:40.505321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340568e-a0dd-45e4-9674-de6ffd480f4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:40.506946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340568e-a0dd-45e4-9674-de6ffd480f4f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:40.512875 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 27/56] 10.222.0.17 () {56 vars in 1322 bytes} [Fri Sep 9 13:00:40 2022] GET /identity/v3/role_assignments?user.id=a4f7da3ad6214b7b94cd5b55e583b917&scope.domain.id=default&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:00:40.569768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8aedf4-a3e7-4736-927a-b3d383776810 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:40.607927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2b26f-982c-489f-8e95-7dcdecfe69b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:40.608568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2b26f-982c-489f-8e95-7dcdecfe69b9 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:40.608854 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2b26f-982c-489f-8e95-7dcdecfe69b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:40.609134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2b26f-982c-489f-8e95-7dcdecfe69b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:40.609809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2b26f-982c-489f-8e95-7dcdecfe69b9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:40.611329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2b26f-982c-489f-8e95-7dcdecfe69b9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:40.639415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 30/57] 10.222.0.17 () {60 vars in 1140 bytes} [Fri Sep 9 13:00:40 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:00:42.804702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362fa9e-3fea-4804-a958-0e7b7b6e89ea None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:42.805208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362fa9e-3fea-4804-a958-0e7b7b6e89ea None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:42.805294 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362fa9e-3fea-4804-a958-0e7b7b6e89ea None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:42.806578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 28/58] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:42.811634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462727c-d88a-493f-b59f-9a38f3ad1903 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:42.812022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462727c-d88a-493f-b59f-9a38f3ad1903 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:42.812189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462727c-d88a-493f-b59f-9a38f3ad1903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:42.831810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b462727c-d88a-493f-b59f-9a38f3ad1903 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:42.832991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b462727c-d88a-493f-b59f-9a38f3ad1903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:42.895987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 31/59] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:43.167933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362fa9e-3fea-4804-a958-0e7b7b6e89ea None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:43.204537 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:43.270318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362fa9e-3fea-4804-a958-0e7b7b6e89ea None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:43.303110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3c1bb-c059-46d1-a156-32d6810a761f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:43.303735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3c1bb-c059-46d1-a156-32d6810a761f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:43.304020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3c1bb-c059-46d1-a156-32d6810a761f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:43.304298 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3c1bb-c059-46d1-a156-32d6810a761f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:43.305424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3c1bb-c059-46d1-a156-32d6810a761f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:43.306884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3c1bb-c059-46d1-a156-32d6810a761f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:43.308477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 29/60] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:00:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:43.314786 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462727c-d88a-493f-b59f-9a38f3ad1903 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:43.335730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:43.413042 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462727c-d88a-493f-b59f-9a38f3ad1903 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:43.445997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f1818-e704-42e1-9112-2ce0edf78e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:43.446591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f1818-e704-42e1-9112-2ce0edf78e8e None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:43.446871 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f1818-e704-42e1-9112-2ce0edf78e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:43.447170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f1818-e704-42e1-9112-2ce0edf78e8e None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:43.447810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f1818-e704-42e1-9112-2ce0edf78e8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:43.449505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f1818-e704-42e1-9112-2ce0edf78e8e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:43.477845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 32/61] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:00:43 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:00:46.774362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129ad06-0d65-4965-ba1a-07e0056068f1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:46.774863 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b697fb-b41b-4abe-94ea-f9c1fb122f66 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:46.775261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129ad06-0d65-4965-ba1a-07e0056068f1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:46.775336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b697fb-b41b-4abe-94ea-f9c1fb122f66 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:46.775401 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129ad06-0d65-4965-ba1a-07e0056068f1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:46.775465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b697fb-b41b-4abe-94ea-f9c1fb122f66 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:46.776302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 30/62] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:46.776575 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 33/63] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:00:46.781797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2f26d-e5a6-4150-9c87-12f0f73c8791 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:46.782169 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2f26d-e5a6-4150-9c87-12f0f73c8791 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:46.782396 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2f26d-e5a6-4150-9c87-12f0f73c8791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:46.794021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df4cc2-bc6e-4f60-b56c-8123f27fa797 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:46.794603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df4cc2-bc6e-4f60-b56c-8123f27fa797 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:46.795127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df4cc2-bc6e-4f60-b56c-8123f27fa797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:46.804281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6ef2f26d-e5a6-4150-9c87-12f0f73c8791 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:46.806072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6ef2f26d-e5a6-4150-9c87-12f0f73c8791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:46.826224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-40df4cc2-bc6e-4f60-b56c-8123f27fa797 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:46.827470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-40df4cc2-bc6e-4f60-b56c-8123f27fa797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:46.889059 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 31/64] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:46.957882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 34/65] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:00:47.035117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c45d17-5dbf-4687-8c96-7120912f03d5 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:47.035117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c45d17-5dbf-4687-8c96-7120912f03d5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:47.035117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c45d17-5dbf-4687-8c96-7120912f03d5 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:47.035117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 32/66] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:00:47.040766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec1c2a-3f1f-4bd5-ac04-94301b70ae3b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:47.041066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec1c2a-3f1f-4bd5-ac04-94301b70ae3b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:47.041351 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec1c2a-3f1f-4bd5-ac04-94301b70ae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:47.057670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:47.064221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-83ec1c2a-3f1f-4bd5-ac04-94301b70ae3b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:47.068365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-83ec1c2a-3f1f-4bd5-ac04-94301b70ae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:47.156612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 35/67] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:47.361210 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c45d17-5dbf-4687-8c96-7120912f03d5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:47.385536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fd19b-1c27-428c-8058-fe6485ec6f98 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:47.386189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fd19b-1c27-428c-8058-fe6485ec6f98 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:47.386689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fd19b-1c27-428c-8058-fe6485ec6f98 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:47.388282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 36/68] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:00:47.394323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cea77-e8ad-4ad2-a6f0-864aae82e92b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:47.394875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cea77-e8ad-4ad2-a6f0-864aae82e92b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:47.395405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cea77-e8ad-4ad2-a6f0-864aae82e92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:47.408958 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:47.417980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b13cea77-e8ad-4ad2-a6f0-864aae82e92b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:47.419478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b13cea77-e8ad-4ad2-a6f0-864aae82e92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:47.494413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 37/69] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:00:47.501097 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cea77-e8ad-4ad2-a6f0-864aae82e92b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:47.507825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c45d17-5dbf-4687-8c96-7120912f03d5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:47.553439 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b23e26-4588-4df1-8018-a50beeab21fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:47.554340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b23e26-4588-4df1-8018-a50beeab21fa None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:47.555099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b23e26-4588-4df1-8018-a50beeab21fa None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:47.555388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b23e26-4588-4df1-8018-a50beeab21fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:47.570715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b23e26-4588-4df1-8018-a50beeab21fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:47.576518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-35b23e26-4588-4df1-8018-a50beeab21fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:47.577883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:47.579242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:47.580018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:47.580018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:47.580018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:47.580018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:47.580018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:47.580018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:47.580018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:47.580018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:47.580018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:47.580018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:47.580018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:00:47.580018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:00:47.580018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 33/70] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:00:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:47.580636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b23e26-4588-4df1-8018-a50beeab21fa None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:47.620065 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cea77-e8ad-4ad2-a6f0-864aae82e92b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:47.654277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9bbe0-9dd7-4dfc-8fdf-d42044884399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:47.654915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9bbe0-9dd7-4dfc-8fdf-d42044884399 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:47.655229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9bbe0-9dd7-4dfc-8fdf-d42044884399 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:47.655508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9bbe0-9dd7-4dfc-8fdf-d42044884399 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:47.666147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9bbe0-9dd7-4dfc-8fdf-d42044884399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0cf9bbe0-9dd7-4dfc-8fdf-d42044884399 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:47.670915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:47.672422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:47.673283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:47.673283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:47.673283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:47.673283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:47.673283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:47.673283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:47.673283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:47.673283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:47.673283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:47.673283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:47.673283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:00:47.673283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:00:47.673283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 38/71] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:00:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:47.673912 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9bbe0-9dd7-4dfc-8fdf-d42044884399 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:47.695390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:47.705508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b23e26-4588-4df1-8018-a50beeab21fa None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:47.740591 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d034c-6db3-408d-9988-6b69e3ceb892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:47.741312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d034c-6db3-408d-9988-6b69e3ceb892 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:47.741596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d034c-6db3-408d-9988-6b69e3ceb892 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:47.741969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d034c-6db3-408d-9988-6b69e3ceb892 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:47.752925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d034c-6db3-408d-9988-6b69e3ceb892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ab4d034c-6db3-408d-9988-6b69e3ceb892 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:47.757708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:47.758692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:47.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:47.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:47.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:47.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:47.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:47.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:47.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:47.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:47.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:47.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:47.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:00:47.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:00:47.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 34/72] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:00:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:47.760112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2020ca6-6b48-45e4-8313-1bc7a3363744 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:47.765859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2020ca6-6b48-45e4-8313-1bc7a3363744 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:47.765859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2020ca6-6b48-45e4-8313-1bc7a3363744 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:47.765859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 35/73] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:47.765859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2020ca6-6b48-45e4-8313-1bc7a3363744 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:47.788586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9bbe0-9dd7-4dfc-8fdf-d42044884399 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:47.815161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:47.826168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004fff6-d172-4b9a-b4ac-e6ded99710b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:47.826780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004fff6-d172-4b9a-b4ac-e6ded99710b1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:47.827075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004fff6-d172-4b9a-b4ac-e6ded99710b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:47.827366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004fff6-d172-4b9a-b4ac-e6ded99710b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:47.828134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004fff6-d172-4b9a-b4ac-e6ded99710b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:47.828438 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004fff6-d172-4b9a-b4ac-e6ded99710b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:47.829841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004fff6-d172-4b9a-b4ac-e6ded99710b1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:47.838231 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 39/74] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:00:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:47.839105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da994d-7d08-4c2e-b44d-ed4adeec4fd1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:47.839395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da994d-7d08-4c2e-b44d-ed4adeec4fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:47.839673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da994d-7d08-4c2e-b44d-ed4adeec4fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:47.857712 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c9da994d-7d08-4c2e-b44d-ed4adeec4fd1 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:47.859788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c9da994d-7d08-4c2e-b44d-ed4adeec4fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:47.868406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2020ca6-6b48-45e4-8313-1bc7a3363744 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:47.902011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25890a94-e933-4eb3-926b-cfe7a3b9d227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:47.902622 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25890a94-e933-4eb3-926b-cfe7a3b9d227 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:47.902904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25890a94-e933-4eb3-926b-cfe7a3b9d227 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:47.903214 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25890a94-e933-4eb3-926b-cfe7a3b9d227 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:47.903884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25890a94-e933-4eb3-926b-cfe7a3b9d227 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:47.904192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25890a94-e933-4eb3-926b-cfe7a3b9d227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:47.905569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25890a94-e933-4eb3-926b-cfe7a3b9d227 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:47.913642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 36/75] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:00:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:47.914515 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25890a94-e933-4eb3-926b-cfe7a3b9d227 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:47.928748 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 40/76] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:47.929598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da994d-7d08-4c2e-b44d-ed4adeec4fd1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:47.938264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:48.022310 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25890a94-e933-4eb3-926b-cfe7a3b9d227 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:48.044690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da994d-7d08-4c2e-b44d-ed4adeec4fd1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:48.058563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3beaa5b-0a67-491c-8ca2-668630d053d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:48.059430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3beaa5b-0a67-491c-8ca2-668630d053d3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:48.059917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3beaa5b-0a67-491c-8ca2-668630d053d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:48.060197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3beaa5b-0a67-491c-8ca2-668630d053d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:48.060861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3beaa5b-0a67-491c-8ca2-668630d053d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:48.061173 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3beaa5b-0a67-491c-8ca2-668630d053d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:48.062613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3beaa5b-0a67-491c-8ca2-668630d053d3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:48.063419 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:48.070787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 37/77] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:00:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:48.076572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3beaa5b-0a67-491c-8ca2-668630d053d3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:48.082814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285abb1f-2905-4587-9569-608f117e6dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:48.083489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285abb1f-2905-4587-9569-608f117e6dda None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:48.083775 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285abb1f-2905-4587-9569-608f117e6dda None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:48.084051 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285abb1f-2905-4587-9569-608f117e6dda None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:48.096526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285abb1f-2905-4587-9569-608f117e6dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-285abb1f-2905-4587-9569-608f117e6dda None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:48.101622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:48.102440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:48.103702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:48.103702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:48.103702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:48.103702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:48.103702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:48.103702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:48.103702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:48.103702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:48.103702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:48.103702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:48.103702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:00:48.103702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:00:48.107327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 41/78] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:00:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:48.108413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285abb1f-2905-4587-9569-608f117e6dda None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:48.158686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:48.186574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3beaa5b-0a67-491c-8ca2-668630d053d3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:48.214640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285abb1f-2905-4587-9569-608f117e6dda None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:48.221691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ad57b-0cba-4657-96f3-11e05b124ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:48.222325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ad57b-0cba-4657-96f3-11e05b124ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:48.222604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ad57b-0cba-4657-96f3-11e05b124ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:48.222877 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ad57b-0cba-4657-96f3-11e05b124ae3 None admin] PATH_INFO: `/v3/users/a4f7da3ad6214b7b94cd5b55e583b917` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:48.224034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ad57b-0cba-4657-96f3-11e05b124ae3 None admin] RBAC: Authorizing `identity:get_user(user_id=a4f7da3ad6214b7b94cd5b55e583b917)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:48.225486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ad57b-0cba-4657-96f3-11e05b124ae3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:48.227077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 38/79] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:48 2022] GET /identity/v3/users/a4f7da3ad6214b7b94cd5b55e583b917 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:48.227839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ad57b-0cba-4657-96f3-11e05b124ae3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:48.253025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2e32a-8544-4d1a-ae13-b47ab70db2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:48.253668 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2e32a-8544-4d1a-ae13-b47ab70db2fe None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:48.253950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2e32a-8544-4d1a-ae13-b47ab70db2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:48.254228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2e32a-8544-4d1a-ae13-b47ab70db2fe None admin] PATH_INFO: `/v3/users/9f28b55f14b8488a9f23c276a094a5ad` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:48.269013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2e32a-8544-4d1a-ae13-b47ab70db2fe None admin] RBAC: Authorizing `identity:get_user(user_id=9f28b55f14b8488a9f23c276a094a5ad)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:48.270483 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2e32a-8544-4d1a-ae13-b47ab70db2fe None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:48.272201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 42/80] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:48 2022] GET /identity/v3/users/9f28b55f14b8488a9f23c276a094a5ad => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:48.272922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2e32a-8544-4d1a-ae13-b47ab70db2fe None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:48.309321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:48.334339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ad57b-0cba-4657-96f3-11e05b124ae3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:48.367901 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10335ca-7198-460e-8810-56cde930d078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:48.368530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10335ca-7198-460e-8810-56cde930d078 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:48.368811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10335ca-7198-460e-8810-56cde930d078 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:48.369083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10335ca-7198-460e-8810-56cde930d078 None admin] PATH_INFO: `/v3/users/9f28b55f14b8488a9f23c276a094a5ad` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:48.370217 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10335ca-7198-460e-8810-56cde930d078 None admin] RBAC: Authorizing `identity:get_user(user_id=9f28b55f14b8488a9f23c276a094a5ad)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:48.371683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10335ca-7198-460e-8810-56cde930d078 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:48.378530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 39/81] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:48 2022] GET /identity/v3/users/9f28b55f14b8488a9f23c276a094a5ad => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:48.378530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10335ca-7198-460e-8810-56cde930d078 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:48.380923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2e32a-8544-4d1a-ae13-b47ab70db2fe None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:48.389188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:48.415622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70ad2f-f71f-4e2d-a50c-1195f539809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:48.416231 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ad2f-f71f-4e2d-a50c-1195f539809e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:48.416510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ad2f-f71f-4e2d-a50c-1195f539809e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:48.416784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ad2f-f71f-4e2d-a50c-1195f539809e None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:48.417438 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70ad2f-f71f-4e2d-a50c-1195f539809e None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:48.417741 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70ad2f-f71f-4e2d-a50c-1195f539809e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:48.419142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70ad2f-f71f-4e2d-a50c-1195f539809e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:48.427680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 43/82] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:00:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:48.433071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70ad2f-f71f-4e2d-a50c-1195f539809e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:48.474197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10335ca-7198-460e-8810-56cde930d078 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:48.508605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e33ee6-70a3-406a-9bca-a34b107ffa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:48.509239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e33ee6-70a3-406a-9bca-a34b107ffa4e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:48.509526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e33ee6-70a3-406a-9bca-a34b107ffa4e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:48.509803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e33ee6-70a3-406a-9bca-a34b107ffa4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:48.510927 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e33ee6-70a3-406a-9bca-a34b107ffa4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:48.512411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e33ee6-70a3-406a-9bca-a34b107ffa4e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:48.513996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 40/83] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:00:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:48.519325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e33ee6-70a3-406a-9bca-a34b107ffa4e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:48.527778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:48.532369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70ad2f-f71f-4e2d-a50c-1195f539809e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:48.565990 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd75d14-40ab-4f02-a1d0-a220ef3a3f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:48.566601 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd75d14-40ab-4f02-a1d0-a220ef3a3f12 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:48.566883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd75d14-40ab-4f02-a1d0-a220ef3a3f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:48.567268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd75d14-40ab-4f02-a1d0-a220ef3a3f12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:48.605408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:48.627653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e33ee6-70a3-406a-9bca-a34b107ffa4e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:48.635759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd75d14-40ab-4f02-a1d0-a220ef3a3f12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:48.637272 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd75d14-40ab-4f02-a1d0-a220ef3a3f12 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:48.660540 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354ab2b-7243-4e70-967e-3b55cd743c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:48.661154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354ab2b-7243-4e70-967e-3b55cd743c9e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:48.661433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354ab2b-7243-4e70-967e-3b55cd743c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:48.661709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354ab2b-7243-4e70-967e-3b55cd743c9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:48.729716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354ab2b-7243-4e70-967e-3b55cd743c9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:48.731082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354ab2b-7243-4e70-967e-3b55cd743c9e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:48.737650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:48.743023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 44/84] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:00:48.744290 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd75d14-40ab-4f02-a1d0-a220ef3a3f12 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:48.817574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:48.838097 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 41/85] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:00:48.838920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354ab2b-7243-4e70-967e-3b55cd743c9e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:48.841944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd75d14-40ab-4f02-a1d0-a220ef3a3f12 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:48.875030 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f93df-eb4e-4c03-a795-c33846da01be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:48.875670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f93df-eb4e-4c03-a795-c33846da01be None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:48.875954 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f93df-eb4e-4c03-a795-c33846da01be None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:48.876228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f93df-eb4e-4c03-a795-c33846da01be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:48.935832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354ab2b-7243-4e70-967e-3b55cd743c9e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:48.939219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:48.943625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f93df-eb4e-4c03-a795-c33846da01be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:48.945092 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f93df-eb4e-4c03-a795-c33846da01be None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:48.970627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fec7b-8bcd-4021-a496-8fdb11416767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:48.971293 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fec7b-8bcd-4021-a496-8fdb11416767 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:48.971576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fec7b-8bcd-4021-a496-8fdb11416767 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:48.971853 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fec7b-8bcd-4021-a496-8fdb11416767 None admin] PATH_INFO: `/v3/users/9f28b55f14b8488a9f23c276a094a5ad` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:48.972989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fec7b-8bcd-4021-a496-8fdb11416767 None admin] RBAC: Authorizing `identity:get_user(user_id=9f28b55f14b8488a9f23c276a094a5ad)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:48.974444 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fec7b-8bcd-4021-a496-8fdb11416767 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:48.976059 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 42/86] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:48 2022] GET /identity/v3/users/9f28b55f14b8488a9f23c276a094a5ad => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:48.976808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fec7b-8bcd-4021-a496-8fdb11416767 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:49.025792 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:49.043393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 45/87] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:00:49.044252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f93df-eb4e-4c03-a795-c33846da01be None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:49.077156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fec7b-8bcd-4021-a496-8fdb11416767 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:49.111061 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fe32f-0b26-4dbc-a627-7f2f40fbbfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:49.111684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fe32f-0b26-4dbc-a627-7f2f40fbbfb8 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:49.111969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fe32f-0b26-4dbc-a627-7f2f40fbbfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:49.112251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fe32f-0b26-4dbc-a627-7f2f40fbbfb8 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:49.112909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fe32f-0b26-4dbc-a627-7f2f40fbbfb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:49.114371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fe32f-0b26-4dbc-a627-7f2f40fbbfb8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:49.138403 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:49.141879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 43/88] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:00:48 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:00:49.142498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f93df-eb4e-4c03-a795-c33846da01be None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:49.144116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fe32f-0b26-4dbc-a627-7f2f40fbbfb8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:49.180996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e38b6-f230-4ff3-a09d-b3dd9be86c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:49.181662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e38b6-f230-4ff3-a09d-b3dd9be86c64 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:49.181942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e38b6-f230-4ff3-a09d-b3dd9be86c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:49.182218 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e38b6-f230-4ff3-a09d-b3dd9be86c64 None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:49.183443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e38b6-f230-4ff3-a09d-b3dd9be86c64 None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:49.184911 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e38b6-f230-4ff3-a09d-b3dd9be86c64 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:49.186594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 46/89] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:00:49 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:49.192523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e38b6-f230-4ff3-a09d-b3dd9be86c64 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:49.261926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fe32f-0b26-4dbc-a627-7f2f40fbbfb8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:49.266505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:49.305640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e38b6-f230-4ff3-a09d-b3dd9be86c64 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:49.309548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd41adb-b6b0-4f45-aee3-22b45edc92c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:49.310201 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41adb-b6b0-4f45-aee3-22b45edc92c9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:49.310478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41adb-b6b0-4f45-aee3-22b45edc92c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:49.310750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41adb-b6b0-4f45-aee3-22b45edc92c9 None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:49.312571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd41adb-b6b0-4f45-aee3-22b45edc92c9 None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:49.314245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd41adb-b6b0-4f45-aee3-22b45edc92c9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:49.321339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 44/90] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:00:49 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:49.321339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd41adb-b6b0-4f45-aee3-22b45edc92c9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:49.346146 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063aa685-2d74-4d84-9696-d9c782799a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:49.346761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aa685-2d74-4d84-9696-d9c782799a87 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:49.347041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aa685-2d74-4d84-9696-d9c782799a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:49.347350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063aa685-2d74-4d84-9696-d9c782799a87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:49.387548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:49.433775 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063aa685-2d74-4d84-9696-d9c782799a87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:49.435532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063aa685-2d74-4d84-9696-d9c782799a87 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:49.442324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd41adb-b6b0-4f45-aee3-22b45edc92c9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:49.482048 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb981f-0048-476e-ad2f-d86bcb59aa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:49.482846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb981f-0048-476e-ad2f-d86bcb59aa2f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:49.483282 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb981f-0048-476e-ad2f-d86bcb59aa2f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:49.483696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb981f-0048-476e-ad2f-d86bcb59aa2f None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:49.485514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb981f-0048-476e-ad2f-d86bcb59aa2f None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:49.486636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb981f-0048-476e-ad2f-d86bcb59aa2f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:49.488474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 45/91] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:00:49 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:49.489366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb981f-0048-476e-ad2f-d86bcb59aa2f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:49.542462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:49.572759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 47/92] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:00:49.575042 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063aa685-2d74-4d84-9696-d9c782799a87 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:49.613135 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb981f-0048-476e-ad2f-d86bcb59aa2f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:49.650541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99895c-bbf3-4125-9822-fa636b926fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:49.651198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99895c-bbf3-4125-9822-fa636b926fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:49.651482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99895c-bbf3-4125-9822-fa636b926fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:49.651761 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99895c-bbf3-4125-9822-fa636b926fd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:49.652496 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99895c-bbf3-4125-9822-fa636b926fd8 None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=482e7368e31f415faa7b8b65d04ae50f, user.id=a4f7da3ad6214b7b94cd5b55e583b917) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:49.652806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99895c-bbf3-4125-9822-fa636b926fd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:49.654317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99895c-bbf3-4125-9822-fa636b926fd8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:49.660274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 46/93] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:00:49 2022] GET /identity/v3/role_assignments?user.id=a4f7da3ad6214b7b94cd5b55e583b917&scope.project.id=482e7368e31f415faa7b8b65d04ae50f&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:49.661157 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99895c-bbf3-4125-9822-fa636b926fd8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:49.666452 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:49.722261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063aa685-2d74-4d84-9696-d9c782799a87 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:49.771540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f717ce6-33ac-4f41-bbdf-6f489c9a6f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:49.772887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f717ce6-33ac-4f41-bbdf-6f489c9a6f92 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:49.773898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f717ce6-33ac-4f41-bbdf-6f489c9a6f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:49.774658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f717ce6-33ac-4f41-bbdf-6f489c9a6f92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:49.775681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f717ce6-33ac-4f41-bbdf-6f489c9a6f92 None admin] RBAC: Adding query filter params (role.id=e7b95eb3b5ad4479b50aaa68e58591ea, scope.project.id=482e7368e31f415faa7b8b65d04ae50f, user.id=9f28b55f14b8488a9f23c276a094a5ad) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:49.776520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f717ce6-33ac-4f41-bbdf-6f489c9a6f92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:49.779142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f717ce6-33ac-4f41-bbdf-6f489c9a6f92 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:49.784893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99895c-bbf3-4125-9822-fa636b926fd8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:49.788915 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:49.792903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 48/94] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:00:49 2022] GET /identity/v3/role_assignments?user.id=9f28b55f14b8488a9f23c276a094a5ad&scope.project.id=482e7368e31f415faa7b8b65d04ae50f&role.id=e7b95eb3b5ad4479b50aaa68e58591ea => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:49.794004 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f717ce6-33ac-4f41-bbdf-6f489c9a6f92 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:49.868036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b02d2-c522-4698-bfcd-a47d55c45d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:49.868689 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b02d2-c522-4698-bfcd-a47d55c45d68 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:49.869076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b02d2-c522-4698-bfcd-a47d55c45d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:49.869373 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b02d2-c522-4698-bfcd-a47d55c45d68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:49.870223 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b02d2-c522-4698-bfcd-a47d55c45d68 None admin] RBAC: Adding query filter params (role.id=0238ab71fb17471cb87e84097272c752, scope.project.id=482e7368e31f415faa7b8b65d04ae50f, user.id=9f28b55f14b8488a9f23c276a094a5ad) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:49.870605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b02d2-c522-4698-bfcd-a47d55c45d68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:49.876447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b02d2-c522-4698-bfcd-a47d55c45d68 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:49.886758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 47/95] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:00:49 2022] GET /identity/v3/role_assignments?user.id=9f28b55f14b8488a9f23c276a094a5ad&scope.project.id=482e7368e31f415faa7b8b65d04ae50f&role.id=0238ab71fb17471cb87e84097272c752 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:49.926943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f717ce6-33ac-4f41-bbdf-6f489c9a6f92 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:49.966282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:49.978710 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f210a2-caa8-441d-bf6b-f19435f1122f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:49.979764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f210a2-caa8-441d-bf6b-f19435f1122f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:49.980230 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f210a2-caa8-441d-bf6b-f19435f1122f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:49.980705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f210a2-caa8-441d-bf6b-f19435f1122f None admin] PATH_INFO: `/v3/projects/dee470142b184e6d8d9bef5b9e1bdfae` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:49.982334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f210a2-caa8-441d-bf6b-f19435f1122f None admin] RBAC: Authorizing `identity:get_project(project_id=dee470142b184e6d8d9bef5b9e1bdfae)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:49.984295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f210a2-caa8-441d-bf6b-f19435f1122f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:49.986519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 49/96] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:00:49 2022] GET /identity/v3/projects/dee470142b184e6d8d9bef5b9e1bdfae => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:49.996260 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b02d2-c522-4698-bfcd-a47d55c45d68 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:50.144441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b02d2-c522-4698-bfcd-a47d55c45d68 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:50.188229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d91f5-473e-4255-b8bd-fae756253b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:50.188852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d91f5-473e-4255-b8bd-fae756253b28 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:50.189134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d91f5-473e-4255-b8bd-fae756253b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:50.189408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d91f5-473e-4255-b8bd-fae756253b28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:50.190144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d91f5-473e-4255-b8bd-fae756253b28 None admin] RBAC: Adding query filter params (role.id=e7b95eb3b5ad4479b50aaa68e58591ea, scope.project.id=dee470142b184e6d8d9bef5b9e1bdfae, user.id=9f28b55f14b8488a9f23c276a094a5ad) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:50.190457 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d91f5-473e-4255-b8bd-fae756253b28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:50.191970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d91f5-473e-4255-b8bd-fae756253b28 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:50.200645 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 48/97] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:00:49 2022] GET /identity/v3/role_assignments?user.id=9f28b55f14b8488a9f23c276a094a5ad&scope.project.id=dee470142b184e6d8d9bef5b9e1bdfae&role.id=e7b95eb3b5ad4479b50aaa68e58591ea => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:52.134192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096125c7-f1a8-4652-a61e-688fb1ed8af0 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:52.134700 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096125c7-f1a8-4652-a61e-688fb1ed8af0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:52.134800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096125c7-f1a8-4652-a61e-688fb1ed8af0 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:52.136159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 50/98] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:52.149113 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb1fc5-1123-4aeb-9b8f-6783ce04aa28 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:52.149570 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb1fc5-1123-4aeb-9b8f-6783ce04aa28 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:52.149751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb1fc5-1123-4aeb-9b8f-6783ce04aa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:52.169226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-47fb1fc5-1123-4aeb-9b8f-6783ce04aa28 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:52.170418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-47fb1fc5-1123-4aeb-9b8f-6783ce04aa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:52.219283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:52.272774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 49/99] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:52.396214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82498726-8496-4f07-82e8-176fc3efcda3 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:52.396501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82498726-8496-4f07-82e8-176fc3efcda3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:52.396690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82498726-8496-4f07-82e8-176fc3efcda3 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:52.398036 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 51/100] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:52.412691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714dd004-7bb2-40dc-bb41-a10804479c40 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:52.412984 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714dd004-7bb2-40dc-bb41-a10804479c40 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:52.413266 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714dd004-7bb2-40dc-bb41-a10804479c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:52.440428 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-714dd004-7bb2-40dc-bb41-a10804479c40 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:52.441686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-714dd004-7bb2-40dc-bb41-a10804479c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:52.527453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 50/101] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:00:52.753563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82498726-8496-4f07-82e8-176fc3efcda3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:52.784060 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52b0fc-4281-4457-b74d-d60f866c85ea None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:52.784689 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52b0fc-4281-4457-b74d-d60f866c85ea None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:52.785184 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52b0fc-4281-4457-b74d-d60f866c85ea None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:52.786748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 51/102] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:52.791941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f6885-d4cf-4aad-9930-930346fe36f6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:52.792474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f6885-d4cf-4aad-9930-930346fe36f6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:52.792951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f6885-d4cf-4aad-9930-930346fe36f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:52.828391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5a4f6885-d4cf-4aad-9930-930346fe36f6 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:52.829972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5a4f6885-d4cf-4aad-9930-930346fe36f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:52.927039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82498726-8496-4f07-82e8-176fc3efcda3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:52.977539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 52/103] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:00:52.979100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f6885-d4cf-4aad-9930-930346fe36f6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:52.984851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138322b-09a1-44b7-aa3e-29f3b990d3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:52.985696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138322b-09a1-44b7-aa3e-29f3b990d3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:52.986818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138322b-09a1-44b7-aa3e-29f3b990d3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:52.987258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138322b-09a1-44b7-aa3e-29f3b990d3a9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:53.011611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138322b-09a1-44b7-aa3e-29f3b990d3a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6138322b-09a1-44b7-aa3e-29f3b990d3a9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:53.018474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:53.019570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:53.020539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:53.020539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:53.020539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:53.020539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:53.020539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:53.020539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:53.020539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:53.020539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:53.020539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:53.020539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:53.020539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:00:53.020539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:00:53.031784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 52/104] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:00:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:53.033733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:53.040312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138322b-09a1-44b7-aa3e-29f3b990d3a9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:53.140198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:53.180758 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138322b-09a1-44b7-aa3e-29f3b990d3a9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:53.208233 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f6885-d4cf-4aad-9930-930346fe36f6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:53.231096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b747b-2777-499e-a156-f9da74672975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:53.232120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b747b-2777-499e-a156-f9da74672975 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:53.232542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b747b-2777-499e-a156-f9da74672975 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:53.233017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b747b-2777-499e-a156-f9da74672975 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:53.234221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b747b-2777-499e-a156-f9da74672975 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:53.234666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b747b-2777-499e-a156-f9da74672975 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:53.241226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b747b-2777-499e-a156-f9da74672975 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:53.262604 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 53/105] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:00:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:53.266578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594370d3-7241-4bc3-89f2-30cc0986fa6c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:53.267541 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594370d3-7241-4bc3-89f2-30cc0986fa6c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:53.269889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594370d3-7241-4bc3-89f2-30cc0986fa6c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:53.270637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8891c-7d6d-453f-ae4d-f478355b4529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:53.271449 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8891c-7d6d-453f-ae4d-f478355b4529 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:53.272859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 54/106] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:53.273292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8891c-7d6d-453f-ae4d-f478355b4529 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:53.273705 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8891c-7d6d-453f-ae4d-f478355b4529 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:53.279624 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8891c-7d6d-453f-ae4d-f478355b4529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:53.281695 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8891c-7d6d-453f-ae4d-f478355b4529 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:53.281834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594370d3-7241-4bc3-89f2-30cc0986fa6c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:53.283994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 53/107] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:00:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:53.285735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daac9d7-4113-47e8-999a-623ea16d143f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:53.294241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daac9d7-4113-47e8-999a-623ea16d143f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:53.294756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daac9d7-4113-47e8-999a-623ea16d143f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:53.329563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0daac9d7-4113-47e8-999a-623ea16d143f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:53.331464 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0daac9d7-4113-47e8-999a-623ea16d143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:53.435901 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 54/108] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:53.437166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daac9d7-4113-47e8-999a-623ea16d143f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:53.443315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594370d3-7241-4bc3-89f2-30cc0986fa6c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:53.461846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:53.485740 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:53.493335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ed6ef-2479-4dce-8403-d23ff878c856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:53.494179 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ed6ef-2479-4dce-8403-d23ff878c856 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:53.494685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ed6ef-2479-4dce-8403-d23ff878c856 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:53.495221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ed6ef-2479-4dce-8403-d23ff878c856 None admin] PATH_INFO: `/v3/users/2e955f68f3a445d4b5a9329b9d2c198c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:53.511770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ed6ef-2479-4dce-8403-d23ff878c856 None admin] RBAC: Authorizing `identity:get_user(user_id=2e955f68f3a445d4b5a9329b9d2c198c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:53.513653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ed6ef-2479-4dce-8403-d23ff878c856 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:53.515726 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 55/109] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:53 2022] GET /identity/v3/users/2e955f68f3a445d4b5a9329b9d2c198c => generated 289 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:53.516910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ed6ef-2479-4dce-8403-d23ff878c856 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:53.593894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daac9d7-4113-47e8-999a-623ea16d143f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:53.633231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc58ded-f07f-430f-bbb8-3157a25d8ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:53.633845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc58ded-f07f-430f-bbb8-3157a25d8ddb None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:53.634121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc58ded-f07f-430f-bbb8-3157a25d8ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:53.634388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc58ded-f07f-430f-bbb8-3157a25d8ddb None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:53.634478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ed6ef-2479-4dce-8403-d23ff878c856 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:53.635030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc58ded-f07f-430f-bbb8-3157a25d8ddb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:53.636511 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc58ded-f07f-430f-bbb8-3157a25d8ddb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:53.669046 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 55/110] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:00:53 2022] POST /identity/v3/projects => generated 302 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:00:53.670912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc58ded-f07f-430f-bbb8-3157a25d8ddb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:53.674806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4865c8-c496-4dda-8814-a99a94b75a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:53.675627 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4865c8-c496-4dda-8814-a99a94b75a77 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:53.677153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4865c8-c496-4dda-8814-a99a94b75a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:53.677636 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4865c8-c496-4dda-8814-a99a94b75a77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:53.694953 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4865c8-c496-4dda-8814-a99a94b75a77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0e4865c8-c496-4dda-8814-a99a94b75a77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:53.702338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:53.703633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:53.704619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:53.704619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:53.704619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:53.704619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:53.704619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:53.704619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:53.704619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:53.704619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:53.704619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:53.704619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:53.704619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:00:53.704619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:00:53.706198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 56/111] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:00:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:53.707696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cf6b4-7e38-44c4-a9eb-cbe35e7a37ae None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:53.708831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cf6b4-7e38-44c4-a9eb-cbe35e7a37ae None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:53.709217 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cf6b4-7e38-44c4-a9eb-cbe35e7a37ae None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:53.715515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 57/112] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:53.717860 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8374e8-5d3b-458d-af9e-55967f696ae7 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:53.718278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8374e8-5d3b-458d-af9e-55967f696ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:53.718798 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8374e8-5d3b-458d-af9e-55967f696ae7 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:53.725679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 58/113] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:53.725679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8374e8-5d3b-458d-af9e-55967f696ae7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:53.772690 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:53.776412 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:53.806578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc58ded-f07f-430f-bbb8-3157a25d8ddb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:53.845760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a297ee-f8fd-4738-88be-c3116baf602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:53.846391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a297ee-f8fd-4738-88be-c3116baf602a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:53.846671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a297ee-f8fd-4738-88be-c3116baf602a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:53.846942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a297ee-f8fd-4738-88be-c3116baf602a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:53.850217 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8374e8-5d3b-458d-af9e-55967f696ae7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:53.887904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466924c5-92d0-4c7e-83ec-f3fec140c2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:53.888533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466924c5-92d0-4c7e-83ec-f3fec140c2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:53.888816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466924c5-92d0-4c7e-83ec-f3fec140c2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:53.889231 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466924c5-92d0-4c7e-83ec-f3fec140c2b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:53.889912 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466924c5-92d0-4c7e-83ec-f3fec140c2b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:53.890216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466924c5-92d0-4c7e-83ec-f3fec140c2b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:53.891653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466924c5-92d0-4c7e-83ec-f3fec140c2b0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:53.900436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 59/114] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:00:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:53.901651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180d4d3-d879-456d-bb89-21e7658e639e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:53.902013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180d4d3-d879-456d-bb89-21e7658e639e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:53.902210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180d4d3-d879-456d-bb89-21e7658e639e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:53.924343 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a297ee-f8fd-4738-88be-c3116baf602a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:53.925819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a297ee-f8fd-4738-88be-c3116baf602a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:53.929996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f180d4d3-d879-456d-bb89-21e7658e639e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:53.931742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f180d4d3-d879-456d-bb89-21e7658e639e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:54.002998 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:54.004647 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 60/115] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:54.006059 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debe09b-41dd-45c0-a18c-99c500945601 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:54.006345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debe09b-41dd-45c0-a18c-99c500945601 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:54.006618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debe09b-41dd-45c0-a18c-99c500945601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:54.026244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6debe09b-41dd-45c0-a18c-99c500945601 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:54.027397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6debe09b-41dd-45c0-a18c-99c500945601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:54.051516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 56/116] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:00:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:00:54.052395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a297ee-f8fd-4738-88be-c3116baf602a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:54.058901 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:54.110478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 61/117] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:54.112307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debe09b-41dd-45c0-a18c-99c500945601 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:54.179527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a297ee-f8fd-4738-88be-c3116baf602a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:54.236347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872c3ef-c5c0-4c95-91fa-8d57f0135780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:54.237681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872c3ef-c5c0-4c95-91fa-8d57f0135780 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:54.238103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872c3ef-c5c0-4c95-91fa-8d57f0135780 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:54.238516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872c3ef-c5c0-4c95-91fa-8d57f0135780 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:54.254388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872c3ef-c5c0-4c95-91fa-8d57f0135780 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:54.256597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debe09b-41dd-45c0-a18c-99c500945601 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9872c3ef-c5c0-4c95-91fa-8d57f0135780 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:54.263336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:54.265091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:54.266549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:54.266549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:54.266549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:54.266549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:54.266549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:54.266549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:54.266549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:54.266549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:54.266549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:54.266549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:54.266549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:00:54.266549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:00:54.270623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 57/118] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:00:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:54.271749 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872c3ef-c5c0-4c95-91fa-8d57f0135780 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:54.309370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:54.333851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805b9cc-0747-44b5-aa19-2b0a8d63dbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:54.334502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805b9cc-0747-44b5-aa19-2b0a8d63dbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:54.334787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805b9cc-0747-44b5-aa19-2b0a8d63dbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:54.335081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805b9cc-0747-44b5-aa19-2b0a8d63dbb0 None admin] PATH_INFO: `/v3/users/9f28b55f14b8488a9f23c276a094a5ad` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:54.344748 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805b9cc-0747-44b5-aa19-2b0a8d63dbb0 None admin] RBAC: Authorizing `identity:get_user(user_id=9f28b55f14b8488a9f23c276a094a5ad)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:54.345985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805b9cc-0747-44b5-aa19-2b0a8d63dbb0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:54.347943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 62/119] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:54 2022] GET /identity/v3/users/9f28b55f14b8488a9f23c276a094a5ad => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:54.348724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805b9cc-0747-44b5-aa19-2b0a8d63dbb0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:54.384175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872c3ef-c5c0-4c95-91fa-8d57f0135780 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:54.389390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:54.419920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c639b-8c82-448c-a11b-0d51f9479e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:54.421826 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c639b-8c82-448c-a11b-0d51f9479e56 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:54.422175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c639b-8c82-448c-a11b-0d51f9479e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:54.422513 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c639b-8c82-448c-a11b-0d51f9479e56 None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:54.423745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c639b-8c82-448c-a11b-0d51f9479e56 None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:54.425290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c639b-8c82-448c-a11b-0d51f9479e56 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:54.427097 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 58/120] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:00:54 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:54.432494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c639b-8c82-448c-a11b-0d51f9479e56 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:54.467967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805b9cc-0747-44b5-aa19-2b0a8d63dbb0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:54.505649 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a7353-a2d4-4905-a49a-d724a13b1517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:54.506770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a7353-a2d4-4905-a49a-d724a13b1517 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:54.507081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a7353-a2d4-4905-a49a-d724a13b1517 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:54.507366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a7353-a2d4-4905-a49a-d724a13b1517 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:54.508041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a7353-a2d4-4905-a49a-d724a13b1517 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:54.508347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a7353-a2d4-4905-a49a-d724a13b1517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:54.509715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a7353-a2d4-4905-a49a-d724a13b1517 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:54.519657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 63/121] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:00:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:54.519657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a7353-a2d4-4905-a49a-d724a13b1517 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:54.544782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:54.549320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c639b-8c82-448c-a11b-0d51f9479e56 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:54.587159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957a02c-223e-448f-8c69-839a8488be60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:54.587813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957a02c-223e-448f-8c69-839a8488be60 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:54.588111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957a02c-223e-448f-8c69-839a8488be60 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:54.588363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957a02c-223e-448f-8c69-839a8488be60 None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:54.589717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957a02c-223e-448f-8c69-839a8488be60 None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:54.593102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957a02c-223e-448f-8c69-839a8488be60 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:54.593224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 59/122] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:00:54 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:54.594047 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957a02c-223e-448f-8c69-839a8488be60 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:54.628138 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:54.639872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a7353-a2d4-4905-a49a-d724a13b1517 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:54.678660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629c6c1-a5c4-4de1-86f3-7b697e9f5320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:54.679814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629c6c1-a5c4-4de1-86f3-7b697e9f5320 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:54.680107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629c6c1-a5c4-4de1-86f3-7b697e9f5320 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:54.680381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629c6c1-a5c4-4de1-86f3-7b697e9f5320 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:54.693792 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629c6c1-a5c4-4de1-86f3-7b697e9f5320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-c629c6c1-a5c4-4de1-86f3-7b697e9f5320 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:54.700856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:54.701916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:54.702724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:54.702724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:54.702724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:54.702724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:54.702724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:54.702724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:54.702724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:54.702724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:54.702724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:54.702724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:54.702724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:00:54.702724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:00:54.702724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 64/123] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:00:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:54.703794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629c6c1-a5c4-4de1-86f3-7b697e9f5320 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:54.712840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957a02c-223e-448f-8c69-839a8488be60 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:54.750604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08eb59-9bd1-4ad9-bb60-d9ac73c01333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:54.751304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08eb59-9bd1-4ad9-bb60-d9ac73c01333 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:54.751591 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08eb59-9bd1-4ad9-bb60-d9ac73c01333 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:54.751866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08eb59-9bd1-4ad9-bb60-d9ac73c01333 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:54.762137 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08eb59-9bd1-4ad9-bb60-d9ac73c01333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4b08eb59-9bd1-4ad9-bb60-d9ac73c01333 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:54.767082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:54.767913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 60/124] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:00:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:54.771713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08eb59-9bd1-4ad9-bb60-d9ac73c01333 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:54.802933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:54.825807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629c6c1-a5c4-4de1-86f3-7b697e9f5320 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:54.862096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98344b6-cc92-4e07-8a3d-5a2837f1a9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:54.862620 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98344b6-cc92-4e07-8a3d-5a2837f1a9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:54.862907 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98344b6-cc92-4e07-8a3d-5a2837f1a9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:54.863210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98344b6-cc92-4e07-8a3d-5a2837f1a9f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:54.863935 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98344b6-cc92-4e07-8a3d-5a2837f1a9f4 None admin] RBAC: Adding query filter params (role.id=fb6fc2065aaa436db8a8d1d8c7f60ade, scope.project.id=482e7368e31f415faa7b8b65d04ae50f, user.id=2e955f68f3a445d4b5a9329b9d2c198c) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:54.864246 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98344b6-cc92-4e07-8a3d-5a2837f1a9f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:54.865695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98344b6-cc92-4e07-8a3d-5a2837f1a9f4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:54.871268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 65/125] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:00:54 2022] GET /identity/v3/role_assignments?user.id=2e955f68f3a445d4b5a9329b9d2c198c&scope.project.id=482e7368e31f415faa7b8b65d04ae50f&role.id=fb6fc2065aaa436db8a8d1d8c7f60ade => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:54.871853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98344b6-cc92-4e07-8a3d-5a2837f1a9f4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:54.886973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08eb59-9bd1-4ad9-bb60-d9ac73c01333 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:54.889963 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:54.943609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c5075-79ba-4b68-8403-e9563c986a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:54.944653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c5075-79ba-4b68-8403-e9563c986a17 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:54.945139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c5075-79ba-4b68-8403-e9563c986a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:54.945694 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c5075-79ba-4b68-8403-e9563c986a17 None admin] PATH_INFO: `/v3/users/9f28b55f14b8488a9f23c276a094a5ad` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:54.947569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c5075-79ba-4b68-8403-e9563c986a17 None admin] RBAC: Authorizing `identity:get_user(user_id=9f28b55f14b8488a9f23c276a094a5ad)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:54.949322 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c5075-79ba-4b68-8403-e9563c986a17 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:54.951467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 61/126] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:54 2022] GET /identity/v3/users/9f28b55f14b8488a9f23c276a094a5ad => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:54.953191 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c5075-79ba-4b68-8403-e9563c986a17 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:54.990440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98344b6-cc92-4e07-8a3d-5a2837f1a9f4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:55.029722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f0fc7-d522-4c1b-ab5e-26fa769e7047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:55.030182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f0fc7-d522-4c1b-ab5e-26fa769e7047 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:55.030439 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f0fc7-d522-4c1b-ab5e-26fa769e7047 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:55.030716 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f0fc7-d522-4c1b-ab5e-26fa769e7047 None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f/users/9f28b55f14b8488a9f23c276a094a5ad/roles/0238ab71fb17471cb87e84097272c752` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:55.033031 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f0fc7-d522-4c1b-ab5e-26fa769e7047 None admin] RBAC: Authorizing `identity:create_grant(project_id=482e7368e31f415faa7b8b65d04ae50f, user_id=9f28b55f14b8488a9f23c276a094a5ad, role_id=0238ab71fb17471cb87e84097272c752)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:55.034514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f0fc7-d522-4c1b-ab5e-26fa769e7047 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:55.056155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 66/127] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:00:54 2022] PUT /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f/users/9f28b55f14b8488a9f23c276a094a5ad/roles/0238ab71fb17471cb87e84097272c752 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:00:55.056886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f0fc7-d522-4c1b-ab5e-26fa769e7047 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:55.079468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:55.084001 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c5075-79ba-4b68-8403-e9563c986a17 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:55.165914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed66fb2-1486-467f-93ad-b3a1ab8db2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:55.166515 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed66fb2-1486-467f-93ad-b3a1ab8db2ce None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:55.166803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed66fb2-1486-467f-93ad-b3a1ab8db2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:55.167099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed66fb2-1486-467f-93ad-b3a1ab8db2ce None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:55.167804 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed66fb2-1486-467f-93ad-b3a1ab8db2ce None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:55.168117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed66fb2-1486-467f-93ad-b3a1ab8db2ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:55.169670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed66fb2-1486-467f-93ad-b3a1ab8db2ce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:55.175661 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:55.188222 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 62/128] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:00:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:55.189202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed66fb2-1486-467f-93ad-b3a1ab8db2ce None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:55.223999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f0fc7-d522-4c1b-ab5e-26fa769e7047 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:55.297972 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425604c8-7c5a-4468-8050-ee557dc06fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:55.299180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425604c8-7c5a-4468-8050-ee557dc06fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:55.299485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425604c8-7c5a-4468-8050-ee557dc06fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:55.299767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425604c8-7c5a-4468-8050-ee557dc06fe1 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:55.300752 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425604c8-7c5a-4468-8050-ee557dc06fe1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:55.301069 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425604c8-7c5a-4468-8050-ee557dc06fe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:55.302555 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425604c8-7c5a-4468-8050-ee557dc06fe1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:55.317108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 67/129] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:00:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:55.318045 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425604c8-7c5a-4468-8050-ee557dc06fe1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:55.372690 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed66fb2-1486-467f-93ad-b3a1ab8db2ce None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:55.421617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a950c-ca1f-470a-aa5d-555115243f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:55.422437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a950c-ca1f-470a-aa5d-555115243f5c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:55.422746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a950c-ca1f-470a-aa5d-555115243f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:55.423414 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a950c-ca1f-470a-aa5d-555115243f5c None admin] PATH_INFO: `/v3/projects/dee470142b184e6d8d9bef5b9e1bdfae` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:55.424594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a950c-ca1f-470a-aa5d-555115243f5c None admin] RBAC: Authorizing `identity:get_project(project_id=dee470142b184e6d8d9bef5b9e1bdfae)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:55.426078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a950c-ca1f-470a-aa5d-555115243f5c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:55.427782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 63/130] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:00:55 2022] GET /identity/v3/projects/dee470142b184e6d8d9bef5b9e1bdfae => generated 312 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:55.428663 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a950c-ca1f-470a-aa5d-555115243f5c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:55.455471 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425604c8-7c5a-4468-8050-ee557dc06fe1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:55.474054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:55.520279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72657ca0-584e-411d-8a64-98b0fa1a3e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:55.521297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72657ca0-584e-411d-8a64-98b0fa1a3e70 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:55.521776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72657ca0-584e-411d-8a64-98b0fa1a3e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:55.522247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72657ca0-584e-411d-8a64-98b0fa1a3e70 None admin] PATH_INFO: `/v3/users/9f28b55f14b8488a9f23c276a094a5ad` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:55.523619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72657ca0-584e-411d-8a64-98b0fa1a3e70 None admin] RBAC: Authorizing `identity:get_user(user_id=9f28b55f14b8488a9f23c276a094a5ad)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:55.526309 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72657ca0-584e-411d-8a64-98b0fa1a3e70 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:55.531259 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 68/131] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:55 2022] GET /identity/v3/users/9f28b55f14b8488a9f23c276a094a5ad => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:55.536638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72657ca0-584e-411d-8a64-98b0fa1a3e70 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:55.544001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:55.610472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a950c-ca1f-470a-aa5d-555115243f5c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:55.653284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673a27b-a5cc-476b-8a97-dcbe49cfc37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:55.653983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673a27b-a5cc-476b-8a97-dcbe49cfc37f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:55.654809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673a27b-a5cc-476b-8a97-dcbe49cfc37f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:55.655121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673a27b-a5cc-476b-8a97-dcbe49cfc37f None admin] PATH_INFO: `/v3/users/a4f7da3ad6214b7b94cd5b55e583b917` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:55.656501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673a27b-a5cc-476b-8a97-dcbe49cfc37f None admin] RBAC: Authorizing `identity:get_user(user_id=a4f7da3ad6214b7b94cd5b55e583b917)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:55.658673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673a27b-a5cc-476b-8a97-dcbe49cfc37f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:55.660589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 64/132] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:55 2022] GET /identity/v3/users/a4f7da3ad6214b7b94cd5b55e583b917 => generated 247 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:55.661367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673a27b-a5cc-476b-8a97-dcbe49cfc37f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:55.735922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72657ca0-584e-411d-8a64-98b0fa1a3e70 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:55.797142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e63f6-7d5c-429c-97b5-7bc548c9c400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:55.799135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e63f6-7d5c-429c-97b5-7bc548c9c400 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:55.799726 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e63f6-7d5c-429c-97b5-7bc548c9c400 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:55.801191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e63f6-7d5c-429c-97b5-7bc548c9c400 None admin] PATH_INFO: `/v3/projects/dee470142b184e6d8d9bef5b9e1bdfae/users/9f28b55f14b8488a9f23c276a094a5ad/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:55.804416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e63f6-7d5c-429c-97b5-7bc548c9c400 None admin] RBAC: Authorizing `identity:create_grant(project_id=dee470142b184e6d8d9bef5b9e1bdfae, user_id=9f28b55f14b8488a9f23c276a094a5ad, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:55.806693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e63f6-7d5c-429c-97b5-7bc548c9c400 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:55.811990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:55.825202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673a27b-a5cc-476b-8a97-dcbe49cfc37f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:55.835834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 69/133] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:00:55 2022] PUT /identity/v3/projects/dee470142b184e6d8d9bef5b9e1bdfae/users/9f28b55f14b8488a9f23c276a094a5ad/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:00:55.837332 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e63f6-7d5c-429c-97b5-7bc548c9c400 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:55.913252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6c97e-6a5b-441e-944b-e7baf6306d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:55.916405 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c97e-6a5b-441e-944b-e7baf6306d21 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:55.917189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c97e-6a5b-441e-944b-e7baf6306d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:55.918192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c97e-6a5b-441e-944b-e7baf6306d21 None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:55.921146 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6c97e-6a5b-441e-944b-e7baf6306d21 None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:55.924019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6c97e-6a5b-441e-944b-e7baf6306d21 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:55.928818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 65/134] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:00:55 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:55.935352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6c97e-6a5b-441e-944b-e7baf6306d21 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:55.979460 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:56.064443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e63f6-7d5c-429c-97b5-7bc548c9c400 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:56.101958 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becba05-7f68-4919-981b-ab329d9be3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:56.102878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becba05-7f68-4919-981b-ab329d9be3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:56.103398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becba05-7f68-4919-981b-ab329d9be3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:56.103816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becba05-7f68-4919-981b-ab329d9be3d5 None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:56.105164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becba05-7f68-4919-981b-ab329d9be3d5 None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:56.106799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becba05-7f68-4919-981b-ab329d9be3d5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:56.109663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 70/135] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:00:55 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:56.115571 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becba05-7f68-4919-981b-ab329d9be3d5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:56.149113 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6c97e-6a5b-441e-944b-e7baf6306d21 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:56.199460 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb60e7-160e-4231-b59c-e1b8d3e3b25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:56.200144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb60e7-160e-4231-b59c-e1b8d3e3b25f None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:56.200429 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb60e7-160e-4231-b59c-e1b8d3e3b25f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:56.200709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb60e7-160e-4231-b59c-e1b8d3e3b25f None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f/users/9f28b55f14b8488a9f23c276a094a5ad/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:56.204602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb60e7-160e-4231-b59c-e1b8d3e3b25f None admin] RBAC: Authorizing `identity:create_grant(project_id=482e7368e31f415faa7b8b65d04ae50f, user_id=9f28b55f14b8488a9f23c276a094a5ad, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:56.206061 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb60e7-160e-4231-b59c-e1b8d3e3b25f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:56.222658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 66/136] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:00:55 2022] PUT /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f/users/9f28b55f14b8488a9f23c276a094a5ad/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:00:56.224121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54c617-93f1-4cc9-96c1-4dff7a56609d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:56.224412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54c617-93f1-4cc9-96c1-4dff7a56609d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:56.224692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54c617-93f1-4cc9-96c1-4dff7a56609d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:56.226165 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 67/137] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:56.232418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f7e68-2550-4c6e-98d6-c9c95c1350e7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:56.232725 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f7e68-2550-4c6e-98d6-c9c95c1350e7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:56.232958 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f7e68-2550-4c6e-98d6-c9c95c1350e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:56.254898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ec6f7e68-2550-4c6e-98d6-c9c95c1350e7 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:56.256253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ec6f7e68-2550-4c6e-98d6-c9c95c1350e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:56.273574 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becba05-7f68-4919-981b-ab329d9be3d5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:56.275714 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:56.319163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:56.329411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1368be-8eb9-4d86-8716-95df7ff9c4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:56.330468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1368be-8eb9-4d86-8716-95df7ff9c4b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:56.330928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1368be-8eb9-4d86-8716-95df7ff9c4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:56.331399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1368be-8eb9-4d86-8716-95df7ff9c4b7 None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f/users/a4f7da3ad6214b7b94cd5b55e583b917/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:56.336647 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1368be-8eb9-4d86-8716-95df7ff9c4b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=482e7368e31f415faa7b8b65d04ae50f, user_id=a4f7da3ad6214b7b94cd5b55e583b917, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:56.338415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1368be-8eb9-4d86-8716-95df7ff9c4b7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:56.352966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 68/138] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:00:56.367006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 71/139] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:00:56 2022] PUT /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f/users/a4f7da3ad6214b7b94cd5b55e583b917/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:00:56.801694 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f7e68-2550-4c6e-98d6-c9c95c1350e7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:56.928935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f7e68-2550-4c6e-98d6-c9c95c1350e7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:56.967034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca0fd5-317e-4075-8f09-b4004a1fcb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:56.968060 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca0fd5-317e-4075-8f09-b4004a1fcb13 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:56.968537 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca0fd5-317e-4075-8f09-b4004a1fcb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:56.969029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca0fd5-317e-4075-8f09-b4004a1fcb13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:56.970362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca0fd5-317e-4075-8f09-b4004a1fcb13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:56.972038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca0fd5-317e-4075-8f09-b4004a1fcb13 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:56.974131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 69/140] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:00:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:56.979580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1368be-8eb9-4d86-8716-95df7ff9c4b7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:57.111859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1368be-8eb9-4d86-8716-95df7ff9c4b7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:57.156060 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02ee2b-ca09-43d4-9a36-1df6714c3f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:57.156676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02ee2b-ca09-43d4-9a36-1df6714c3f9a None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:57.156958 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02ee2b-ca09-43d4-9a36-1df6714c3f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:57.157234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02ee2b-ca09-43d4-9a36-1df6714c3f9a None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:57.157899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02ee2b-ca09-43d4-9a36-1df6714c3f9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:57.159381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02ee2b-ca09-43d4-9a36-1df6714c3f9a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:57.195132 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 72/141] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:00:56 2022] POST /identity/v3/users => generated 283 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:00:57.741172 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571ab19-de07-4340-8622-2f26cc346c4f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:57.741560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571ab19-de07-4340-8622-2f26cc346c4f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:57.741751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571ab19-de07-4340-8622-2f26cc346c4f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:57.743305 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 70/142] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:57.750901 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d7e95-7ef7-455f-9b6a-1e0df7439b96 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:57.751841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d7e95-7ef7-455f-9b6a-1e0df7439b96 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:57.752342 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d7e95-7ef7-455f-9b6a-1e0df7439b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:57.773450 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-123d7e95-7ef7-455f-9b6a-1e0df7439b96 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:57.774980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-123d7e95-7ef7-455f-9b6a-1e0df7439b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:57.828724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:57.854578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 73/143] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:58.050035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a3e59-e801-4519-b558-6be32e13533c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:58.050562 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a3e59-e801-4519-b558-6be32e13533c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:58.050562 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a3e59-e801-4519-b558-6be32e13533c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:58.051966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 71/144] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:58.057091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ef2f1-46b3-49f3-b2c4-46454283d446 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:58.057384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ef2f1-46b3-49f3-b2c4-46454283d446 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:58.057659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ef2f1-46b3-49f3-b2c4-46454283d446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:58.084043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-618ef2f1-46b3-49f3-b2c4-46454283d446 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:58.085240 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-618ef2f1-46b3-49f3-b2c4-46454283d446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:58.170888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 74/145] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:58.391361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a3e59-e801-4519-b558-6be32e13533c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:58.431320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:58.520868 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ef2f1-46b3-49f3-b2c4-46454283d446 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:58.592611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a3e59-e801-4519-b558-6be32e13533c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:58.705603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ebef1-301c-4d92-a7b8-6d478c8ac662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:58.706292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ebef1-301c-4d92-a7b8-6d478c8ac662 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:58.706568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ebef1-301c-4d92-a7b8-6d478c8ac662 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:58.706840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ebef1-301c-4d92-a7b8-6d478c8ac662 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:58.736219 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ebef1-301c-4d92-a7b8-6d478c8ac662 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-613ebef1-301c-4d92-a7b8-6d478c8ac662 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:58.744549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:58.745483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:58.746229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:58.746229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:58.746229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:58.746229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:58.746229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:58.746229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:58.746229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:58.746229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:58.746229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:58.746229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:58.746229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:00:58.746229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:00:58.746812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 72/146] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:00:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:58.748925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ef2f1-46b3-49f3-b2c4-46454283d446 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:58.753371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ebef1-301c-4d92-a7b8-6d478c8ac662 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:58.773531 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:58.844678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bfffd-0b3e-4ee2-a942-32aba93c2ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:58.846186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bfffd-0b3e-4ee2-a942-32aba93c2ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:58.851888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bfffd-0b3e-4ee2-a942-32aba93c2ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:58.852425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bfffd-0b3e-4ee2-a942-32aba93c2ff7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:58.882668 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933bfffd-0b3e-4ee2-a942-32aba93c2ff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-933bfffd-0b3e-4ee2-a942-32aba93c2ff7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:00:58.892358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:00:58.893416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:00:58.894224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:00:58.894224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:00:58.894224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:00:58.894224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:00:58.894224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:00:58.894224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:00:58.894224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:00:58.894224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:00:58.894224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:00:58.894224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:00:58.894224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:00:58.894224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:00:58.902110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 75/147] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:00:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:00:58.917772 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bfffd-0b3e-4ee2-a942-32aba93c2ff7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:58.995026 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:59.051942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ebef1-301c-4d92-a7b8-6d478c8ac662 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:59.135589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bfffd-0b3e-4ee2-a942-32aba93c2ff7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:59.160850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb83b6-18d9-4ba5-8449-fc187b81bf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:59.161483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb83b6-18d9-4ba5-8449-fc187b81bf08 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:59.161763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb83b6-18d9-4ba5-8449-fc187b81bf08 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:59.162038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb83b6-18d9-4ba5-8449-fc187b81bf08 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:59.162718 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb83b6-18d9-4ba5-8449-fc187b81bf08 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:59.163019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb83b6-18d9-4ba5-8449-fc187b81bf08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:59.164580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb83b6-18d9-4ba5-8449-fc187b81bf08 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:59.184017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3325a1-78f8-4757-83a1-0c9292735378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:59.184742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3325a1-78f8-4757-83a1-0c9292735378 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:59.185611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3325a1-78f8-4757-83a1-0c9292735378 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:59.185943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3325a1-78f8-4757-83a1-0c9292735378 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:59.186700 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3325a1-78f8-4757-83a1-0c9292735378 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:00:59.187461 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 73/148] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:00:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:59.187942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3325a1-78f8-4757-83a1-0c9292735378 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:59.188733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a49c69-bde6-4059-bedc-a9a6ba45a9c1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:59.189014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a49c69-bde6-4059-bedc-a9a6ba45a9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:59.189286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a49c69-bde6-4059-bedc-a9a6ba45a9c1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:59.189383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3325a1-78f8-4757-83a1-0c9292735378 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:59.190629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 74/149] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:00:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:00:59.197715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a49c69-bde6-4059-bedc-a9a6ba45a9c1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:59.199392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 76/150] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:00:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:59.205613 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bef7e-92fe-4d14-b714-0ebca21e19c3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:59.206020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bef7e-92fe-4d14-b714-0ebca21e19c3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:59.206397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bef7e-92fe-4d14-b714-0ebca21e19c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:59.241868 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0d6bef7e-92fe-4d14-b714-0ebca21e19c3 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:59.246275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0d6bef7e-92fe-4d14-b714-0ebca21e19c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:59.295413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:59.405744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 77/151] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:00:59.408811 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bef7e-92fe-4d14-b714-0ebca21e19c3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:59.446075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a49c69-bde6-4059-bedc-a9a6ba45a9c1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:59.519026 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ebde8-71a1-4383-bb06-057607cce599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:59.519613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ebde8-71a1-4383-bb06-057607cce599 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:59.519890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ebde8-71a1-4383-bb06-057607cce599 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:59.520158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ebde8-71a1-4383-bb06-057607cce599 None admin] PATH_INFO: `/v3/users/2e955f68f3a445d4b5a9329b9d2c198c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:59.522293 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ebde8-71a1-4383-bb06-057607cce599 None admin] RBAC: Authorizing `identity:get_user(user_id=2e955f68f3a445d4b5a9329b9d2c198c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:59.523782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ebde8-71a1-4383-bb06-057607cce599 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:59.525371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 75/152] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:59 2022] GET /identity/v3/users/2e955f68f3a445d4b5a9329b9d2c198c => generated 289 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:59.531980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ebde8-71a1-4383-bb06-057607cce599 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:59.562558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:59.613063 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bef7e-92fe-4d14-b714-0ebca21e19c3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:59.683588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3007fae-b43e-40ad-ade8-2e864c2f25d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:59.684283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3007fae-b43e-40ad-ade8-2e864c2f25d9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:59.684568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3007fae-b43e-40ad-ade8-2e864c2f25d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:59.684959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3007fae-b43e-40ad-ade8-2e864c2f25d9 None admin] PATH_INFO: `/v3/users/9f28b55f14b8488a9f23c276a094a5ad` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:59.686195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3007fae-b43e-40ad-ade8-2e864c2f25d9 None admin] RBAC: Authorizing `identity:get_user(user_id=9f28b55f14b8488a9f23c276a094a5ad)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:59.687920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3007fae-b43e-40ad-ade8-2e864c2f25d9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:59.689760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 78/153] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:00:59 2022] GET /identity/v3/users/9f28b55f14b8488a9f23c276a094a5ad => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:59.695164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3007fae-b43e-40ad-ade8-2e864c2f25d9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:59.723310 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ebde8-71a1-4383-bb06-057607cce599 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:59.760780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:59.783894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcbdb7-bf19-46c8-b938-437ada50904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:59.784467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcbdb7-bf19-46c8-b938-437ada50904a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:59.784743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcbdb7-bf19-46c8-b938-437ada50904a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:59.785010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcbdb7-bf19-46c8-b938-437ada50904a None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:59.786149 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcbdb7-bf19-46c8-b938-437ada50904a None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:59.787706 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcbdb7-bf19-46c8-b938-437ada50904a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:59.790150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 76/154] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:00:59 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:00:59.798076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcbdb7-bf19-46c8-b938-437ada50904a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:59.914553 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3007fae-b43e-40ad-ade8-2e864c2f25d9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:59.935136 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:59.976083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcbdb7-bf19-46c8-b938-437ada50904a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:59.993609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25621ee4-b764-40e2-a1f9-c73419652325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:59.994236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25621ee4-b764-40e2-a1f9-c73419652325 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:59.994515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25621ee4-b764-40e2-a1f9-c73419652325 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:59.994791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25621ee4-b764-40e2-a1f9-c73419652325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:00.018477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62f330-d8e1-48fd-9731-6c0967bd1314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:00.019461 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62f330-d8e1-48fd-9731-6c0967bd1314 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:00.019834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62f330-d8e1-48fd-9731-6c0967bd1314 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:00.020199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62f330-d8e1-48fd-9731-6c0967bd1314 None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f/users/2e955f68f3a445d4b5a9329b9d2c198c/roles/fb6fc2065aaa436db8a8d1d8c7f60ade` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:00.022971 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62f330-d8e1-48fd-9731-6c0967bd1314 None admin] RBAC: Authorizing `identity:create_grant(project_id=482e7368e31f415faa7b8b65d04ae50f, user_id=2e955f68f3a445d4b5a9329b9d2c198c, role_id=fb6fc2065aaa436db8a8d1d8c7f60ade)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:00.024548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62f330-d8e1-48fd-9731-6c0967bd1314 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:00.054090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 77/155] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:00:59 2022] PUT /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f/users/2e955f68f3a445d4b5a9329b9d2c198c/roles/fb6fc2065aaa436db8a8d1d8c7f60ade => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:01:00.120042 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25621ee4-b764-40e2-a1f9-c73419652325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:00.121536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25621ee4-b764-40e2-a1f9-c73419652325 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:00.161147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:00.162891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62f330-d8e1-48fd-9731-6c0967bd1314 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:00.266357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 79/156] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 13:01:00.268319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62ca66-f506-4979-9b96-cba2c1d76f2e None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:00.268767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62ca66-f506-4979-9b96-cba2c1d76f2e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:00.269196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62ca66-f506-4979-9b96-cba2c1d76f2e None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:00.270628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 80/157] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:00.275785 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62ca66-f506-4979-9b96-cba2c1d76f2e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:00.371403 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62f330-d8e1-48fd-9731-6c0967bd1314 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:00.459774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba269d8-0187-4b71-af0e-fca4509e3806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:00.460433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba269d8-0187-4b71-af0e-fca4509e3806 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:00.460710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba269d8-0187-4b71-af0e-fca4509e3806 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:00.460984 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba269d8-0187-4b71-af0e-fca4509e3806 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:00.475679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:00.483142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62ca66-f506-4979-9b96-cba2c1d76f2e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:00.485832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba269d8-0187-4b71-af0e-fca4509e3806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9ba269d8-0187-4b71-af0e-fca4509e3806 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:00.490852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:00.492158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:00.493244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:00.493244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:00.493244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:00.493244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:00.493244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:00.493244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:00.493244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:00.493244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:00.493244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:00.493244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:00.493244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:00.493244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:00.493244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 78/158] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:00.495986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0f3ad-6a3d-4ffc-b01e-abdeca9b33ee None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:00.495986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0f3ad-6a3d-4ffc-b01e-abdeca9b33ee None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:00.495986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0f3ad-6a3d-4ffc-b01e-abdeca9b33ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:00.537372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:00.537558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-dcc0f3ad-6a3d-4ffc-b01e-abdeca9b33ee None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:00.538873 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dcc0f3ad-6a3d-4ffc-b01e-abdeca9b33ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:00.553304 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1ea62-e48b-473c-b863-f837aac0578b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:00.553948 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1ea62-e48b-473c-b863-f837aac0578b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:00.554226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1ea62-e48b-473c-b863-f837aac0578b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:00.554526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1ea62-e48b-473c-b863-f837aac0578b None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:00.557657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1ea62-e48b-473c-b863-f837aac0578b None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:00.560139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1ea62-e48b-473c-b863-f837aac0578b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:00.561710 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 81/159] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:00 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:00.562501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1ea62-e48b-473c-b863-f837aac0578b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:00.677971 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 79/160] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:00.678863 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0f3ad-6a3d-4ffc-b01e-abdeca9b33ee None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:00.680652 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1ea62-e48b-473c-b863-f837aac0578b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:00.728085 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0382e66-e3a7-4301-9d03-87513306a885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:00.728703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0382e66-e3a7-4301-9d03-87513306a885 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:00.728982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0382e66-e3a7-4301-9d03-87513306a885 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:00.729367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0382e66-e3a7-4301-9d03-87513306a885 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:00.730050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0382e66-e3a7-4301-9d03-87513306a885 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:00.730359 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0382e66-e3a7-4301-9d03-87513306a885 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:00.731851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0382e66-e3a7-4301-9d03-87513306a885 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:00.741168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 82/161] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:00.742939 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f988045-516b-4e11-b79a-7490dcc2ca24 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:00.743252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f988045-516b-4e11-b79a-7490dcc2ca24 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:00.743529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f988045-516b-4e11-b79a-7490dcc2ca24 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:00.745193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 83/162] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:00.748671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f988045-516b-4e11-b79a-7490dcc2ca24 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:00.823418 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:00.869902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f988045-516b-4e11-b79a-7490dcc2ca24 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:00.916661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0f3ad-6a3d-4ffc-b01e-abdeca9b33ee None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:00.939630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2d1bf-8fc4-4cfc-a8ad-58e5b0548fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:00.940980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2d1bf-8fc4-4cfc-a8ad-58e5b0548fef None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:00.941268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2d1bf-8fc4-4cfc-a8ad-58e5b0548fef None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:00.941541 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2d1bf-8fc4-4cfc-a8ad-58e5b0548fef None admin] PATH_INFO: `/v3/users/9f28b55f14b8488a9f23c276a094a5ad` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:00.945741 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2d1bf-8fc4-4cfc-a8ad-58e5b0548fef None admin] RBAC: Authorizing `identity:get_user(user_id=9f28b55f14b8488a9f23c276a094a5ad)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:00.948054 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2d1bf-8fc4-4cfc-a8ad-58e5b0548fef None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:00.951955 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 84/163] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:00 2022] GET /identity/v3/users/9f28b55f14b8488a9f23c276a094a5ad => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:00.953158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6ef4e-d03f-4c67-97d6-1622e008fe3d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:00.953442 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6ef4e-d03f-4c67-97d6-1622e008fe3d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:00.953718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6ef4e-d03f-4c67-97d6-1622e008fe3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:00.976913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:00.992865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d2a6ef4e-d03f-4c67-97d6-1622e008fe3d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:00.994131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67008f0a-9adc-48e6-b5bc-f8c3cc8713bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:00.994749 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67008f0a-9adc-48e6-b5bc-f8c3cc8713bb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:00.995023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67008f0a-9adc-48e6-b5bc-f8c3cc8713bb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:00.995323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67008f0a-9adc-48e6-b5bc-f8c3cc8713bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:00.996056 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67008f0a-9adc-48e6-b5bc-f8c3cc8713bb None admin] RBAC: Adding query filter params (role.id=0238ab71fb17471cb87e84097272c752, scope.project.id=482e7368e31f415faa7b8b65d04ae50f, user.id=9f28b55f14b8488a9f23c276a094a5ad) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:00.996361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67008f0a-9adc-48e6-b5bc-f8c3cc8713bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:00.998721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d2a6ef4e-d03f-4c67-97d6-1622e008fe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:01.001378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67008f0a-9adc-48e6-b5bc-f8c3cc8713bb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:01.009712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 80/164] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:00 2022] GET /identity/v3/role_assignments?user.id=9f28b55f14b8488a9f23c276a094a5ad&scope.project.id=482e7368e31f415faa7b8b65d04ae50f&role.id=0238ab71fb17471cb87e84097272c752 => generated 608 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:01.015088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67008f0a-9adc-48e6-b5bc-f8c3cc8713bb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:01.188240 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67008f0a-9adc-48e6-b5bc-f8c3cc8713bb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:01.240893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 85/165] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:01.251297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27036dfd-c876-4ac7-b609-b0de5688cef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:01.251939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27036dfd-c876-4ac7-b609-b0de5688cef8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:01.252215 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27036dfd-c876-4ac7-b609-b0de5688cef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:01.252484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27036dfd-c876-4ac7-b609-b0de5688cef8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:01.328903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6ef4e-d03f-4c67-97d6-1622e008fe3d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:01.341787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:01.349565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27036dfd-c876-4ac7-b609-b0de5688cef8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:01.351082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27036dfd-c876-4ac7-b609-b0de5688cef8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:01.443446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:01.499361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6ef4e-d03f-4c67-97d6-1622e008fe3d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:01.531775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 81/166] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:01.551119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044f6dd-a490-47c0-af43-f567e2292ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:01.551773 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044f6dd-a490-47c0-af43-f567e2292ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:01.552048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044f6dd-a490-47c0-af43-f567e2292ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:01.552150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27036dfd-c876-4ac7-b609-b0de5688cef8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:01.552321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044f6dd-a490-47c0-af43-f567e2292ae5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:01.573014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044f6dd-a490-47c0-af43-f567e2292ae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d044f6dd-a490-47c0-af43-f567e2292ae5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:01.578392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:01.579367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:01.580306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:01.580306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:01.580306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:01.580306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:01.580306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:01.580306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:01.580306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:01.580306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:01.580306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:01.580306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:01.580306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:01.580306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:01.596607 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 86/167] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:01:01.602354 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044f6dd-a490-47c0-af43-f567e2292ae5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:01.699428 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:01.712724 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27036dfd-c876-4ac7-b609-b0de5688cef8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:01.775884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38dcf8d-6940-45a5-bee1-a885a41d00d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:01.776925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38dcf8d-6940-45a5-bee1-a885a41d00d2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:01.779349 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38dcf8d-6940-45a5-bee1-a885a41d00d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:01.780610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38dcf8d-6940-45a5-bee1-a885a41d00d2 None admin] PATH_INFO: `/v3/projects/dee470142b184e6d8d9bef5b9e1bdfae` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:01.782547 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38dcf8d-6940-45a5-bee1-a885a41d00d2 None admin] RBAC: Authorizing `identity:get_project(project_id=dee470142b184e6d8d9bef5b9e1bdfae)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:01.784330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38dcf8d-6940-45a5-bee1-a885a41d00d2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:01.787239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 82/168] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:01 2022] GET /identity/v3/projects/dee470142b184e6d8d9bef5b9e1bdfae => generated 312 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:01.793403 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38dcf8d-6940-45a5-bee1-a885a41d00d2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:01.896964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044f6dd-a490-47c0-af43-f567e2292ae5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:01.951425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:01.982915 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38dcf8d-6940-45a5-bee1-a885a41d00d2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:02.007893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d04c69-97fa-44fe-b6de-2b01705162af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:02.008520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d04c69-97fa-44fe-b6de-2b01705162af None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:02.008798 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d04c69-97fa-44fe-b6de-2b01705162af None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:02.009068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d04c69-97fa-44fe-b6de-2b01705162af None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:02.009744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d04c69-97fa-44fe-b6de-2b01705162af None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:02.010043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d04c69-97fa-44fe-b6de-2b01705162af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:02.011453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d04c69-97fa-44fe-b6de-2b01705162af None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:02.031446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 87/169] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:02.032250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d04c69-97fa-44fe-b6de-2b01705162af None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:02.069078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0956a8-0412-4b12-b72c-a8b6440c7fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:02.070029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0956a8-0412-4b12-b72c-a8b6440c7fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:02.070310 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0956a8-0412-4b12-b72c-a8b6440c7fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:02.070637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0956a8-0412-4b12-b72c-a8b6440c7fd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:02.071658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0956a8-0412-4b12-b72c-a8b6440c7fd9 None admin] RBAC: Adding query filter params (role.id=e7b95eb3b5ad4479b50aaa68e58591ea, scope.project.id=dee470142b184e6d8d9bef5b9e1bdfae, user.id=9f28b55f14b8488a9f23c276a094a5ad) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:02.071967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0956a8-0412-4b12-b72c-a8b6440c7fd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:02.073953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0956a8-0412-4b12-b72c-a8b6440c7fd9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:02.088839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 83/170] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:01 2022] GET /identity/v3/role_assignments?user.id=9f28b55f14b8488a9f23c276a094a5ad&scope.project.id=dee470142b184e6d8d9bef5b9e1bdfae&role.id=e7b95eb3b5ad4479b50aaa68e58591ea => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:02.089785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0956a8-0412-4b12-b72c-a8b6440c7fd9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:02.156745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:02.302039 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0956a8-0412-4b12-b72c-a8b6440c7fd9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:02.308047 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d04c69-97fa-44fe-b6de-2b01705162af None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:02.361747 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c945ee-6279-44ee-a2be-69ce200849c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:02.362413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c945ee-6279-44ee-a2be-69ce200849c6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:02.362692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c945ee-6279-44ee-a2be-69ce200849c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:02.362959 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c945ee-6279-44ee-a2be-69ce200849c6 None admin] PATH_INFO: `/v3/users/9f28b55f14b8488a9f23c276a094a5ad` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:02.364971 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c945ee-6279-44ee-a2be-69ce200849c6 None admin] RBAC: Authorizing `identity:get_user(user_id=9f28b55f14b8488a9f23c276a094a5ad)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:02.366704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c945ee-6279-44ee-a2be-69ce200849c6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:02.368631 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 84/171] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:02 2022] GET /identity/v3/users/9f28b55f14b8488a9f23c276a094a5ad => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:02.369841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20739f99-0aa6-4be9-9d30-2125bb2b98b7 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:02.370293 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20739f99-0aa6-4be9-9d30-2125bb2b98b7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:02.370949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20739f99-0aa6-4be9-9d30-2125bb2b98b7 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:02.373537 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 85/172] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:02.378997 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20739f99-0aa6-4be9-9d30-2125bb2b98b7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:02.399180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b65080-cd9b-4cd4-9b50-9657f1d8ff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:02.400101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b65080-cd9b-4cd4-9b50-9657f1d8ff7e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:02.400532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b65080-cd9b-4cd4-9b50-9657f1d8ff7e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:02.400990 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b65080-cd9b-4cd4-9b50-9657f1d8ff7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:02.423666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b65080-cd9b-4cd4-9b50-9657f1d8ff7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-32b65080-cd9b-4cd4-9b50-9657f1d8ff7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:02.429352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:02.430480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:02.431568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:02.431568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:02.431568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:02.431568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:02.431568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:02.431568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:02.431568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:02.431568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:02.431568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:02.431568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:02.431568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:02.431568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:02.437509 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 88/173] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:01:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:02.439235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d165166-a27f-4f55-9fdf-1f99d98cce36 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:02.441691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d165166-a27f-4f55-9fdf-1f99d98cce36 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:02.442735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d165166-a27f-4f55-9fdf-1f99d98cce36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:02.476348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5d165166-a27f-4f55-9fdf-1f99d98cce36 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:02.477670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5d165166-a27f-4f55-9fdf-1f99d98cce36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:02.513450 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:02.560010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:02.581021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20739f99-0aa6-4be9-9d30-2125bb2b98b7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:02.600082 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 89/174] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:02.602505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d165166-a27f-4f55-9fdf-1f99d98cce36 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:02.659220 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40460c80-f4a9-4389-a007-7fe5094c3208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:02.660865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40460c80-f4a9-4389-a007-7fe5094c3208 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:02.661143 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40460c80-f4a9-4389-a007-7fe5094c3208 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:02.661412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40460c80-f4a9-4389-a007-7fe5094c3208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:02.758408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d165166-a27f-4f55-9fdf-1f99d98cce36 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:02.777962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40460c80-f4a9-4389-a007-7fe5094c3208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:02.779517 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40460c80-f4a9-4389-a007-7fe5094c3208 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:02.847885 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac77cd2-8e27-433c-a76d-d502d21d1f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:02.848439 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac77cd2-8e27-433c-a76d-d502d21d1f94 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:02.848723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac77cd2-8e27-433c-a76d-d502d21d1f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:02.848995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac77cd2-8e27-433c-a76d-d502d21d1f94 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:02.849676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac77cd2-8e27-433c-a76d-d502d21d1f94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:02.849979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac77cd2-8e27-433c-a76d-d502d21d1f94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:02.851398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac77cd2-8e27-433c-a76d-d502d21d1f94 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:02.865900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 90/175] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:02.871868 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac77cd2-8e27-433c-a76d-d502d21d1f94 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:02.944319 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:02.945622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:02.963634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 86/176] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:02.964999 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a17c3-ac16-4c43-8b84-4886c8c48575 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:02.965290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a17c3-ac16-4c43-8b84-4886c8c48575 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:02.965675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a17c3-ac16-4c43-8b84-4886c8c48575 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:02.975263 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 87/177] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:02.975263 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a17c3-ac16-4c43-8b84-4886c8c48575 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:03.066510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac77cd2-8e27-433c-a76d-d502d21d1f94 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:03.105035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6c98c-490c-40a0-8aee-32e3e70c8501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:03.105618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6c98c-490c-40a0-8aee-32e3e70c8501 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:03.105926 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6c98c-490c-40a0-8aee-32e3e70c8501 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:03.106198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6c98c-490c-40a0-8aee-32e3e70c8501 None admin] PATH_INFO: `/v3/users/a4f7da3ad6214b7b94cd5b55e583b917` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:03.107392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6c98c-490c-40a0-8aee-32e3e70c8501 None admin] RBAC: Authorizing `identity:get_user(user_id=a4f7da3ad6214b7b94cd5b55e583b917)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:03.108920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6c98c-490c-40a0-8aee-32e3e70c8501 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:03.110635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 91/178] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:02 2022] GET /identity/v3/users/a4f7da3ad6214b7b94cd5b55e583b917 => generated 247 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:03.111776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde944c8-6b42-4037-b3c6-267212259d6b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:03.112065 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde944c8-6b42-4037-b3c6-267212259d6b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:03.112340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde944c8-6b42-4037-b3c6-267212259d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:03.117447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a17c3-ac16-4c43-8b84-4886c8c48575 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:03.133092 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-bde944c8-6b42-4037-b3c6-267212259d6b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:03.134294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bde944c8-6b42-4037-b3c6-267212259d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:03.156359 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70300782-b887-40a2-b071-56b83254644f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:03.157310 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70300782-b887-40a2-b071-56b83254644f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:03.157310 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70300782-b887-40a2-b071-56b83254644f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:03.157510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70300782-b887-40a2-b071-56b83254644f None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:03.158623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70300782-b887-40a2-b071-56b83254644f None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:03.160179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70300782-b887-40a2-b071-56b83254644f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:03.166452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 88/179] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:02 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:03.166452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70300782-b887-40a2-b071-56b83254644f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:03.213174 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 92/180] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:03.214457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5c4d5-dbf5-4046-9519-fb5762c82bea None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:03.214743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5c4d5-dbf5-4046-9519-fb5762c82bea None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:03.215018 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5c4d5-dbf5-4046-9519-fb5762c82bea None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:03.216580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 93/181] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:03.217130 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5c4d5-dbf5-4046-9519-fb5762c82bea None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:03.247933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:03.256320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:03.292868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70300782-b887-40a2-b071-56b83254644f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:03.329837 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada19d4-1c58-499b-9c01-a4436100f6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:03.330645 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada19d4-1c58-499b-9c01-a4436100f6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:03.331171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada19d4-1c58-499b-9c01-a4436100f6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:03.331740 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada19d4-1c58-499b-9c01-a4436100f6c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:03.344178 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada19d4-1c58-499b-9c01-a4436100f6c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5ada19d4-1c58-499b-9c01-a4436100f6c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:03.349868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:03.350975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:03.351920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:03.351920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:03.351920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:03.351920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:03.351920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:03.351920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:03.351920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:03.351920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:03.351920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:03.351920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:03.351920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:03.351920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:03.353054 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5c4d5-dbf5-4046-9519-fb5762c82bea None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:03.354241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 89/182] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:01:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:03.355458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada19d4-1c58-499b-9c01-a4436100f6c3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:03.398585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1623e-2c0c-4a67-80df-ae3a741b83a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:03.399255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1623e-2c0c-4a67-80df-ae3a741b83a9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:03.399518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1623e-2c0c-4a67-80df-ae3a741b83a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:03.399787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1623e-2c0c-4a67-80df-ae3a741b83a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:03.482027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1623e-2c0c-4a67-80df-ae3a741b83a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:03.483539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1623e-2c0c-4a67-80df-ae3a741b83a9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:03.488464 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada19d4-1c58-499b-9c01-a4436100f6c3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:03.510670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:03.513478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:03.527173 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8ee03-c38e-4c40-9533-9600c679bc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:03.528407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8ee03-c38e-4c40-9533-9600c679bc88 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:03.528821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8ee03-c38e-4c40-9533-9600c679bc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:03.529103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8ee03-c38e-4c40-9533-9600c679bc88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:03.529837 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8ee03-c38e-4c40-9533-9600c679bc88 None admin] RBAC: Adding query filter params (role.id=e7b95eb3b5ad4479b50aaa68e58591ea, scope.project.id=482e7368e31f415faa7b8b65d04ae50f, user.id=9f28b55f14b8488a9f23c276a094a5ad) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:03.530281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8ee03-c38e-4c40-9533-9600c679bc88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:03.531787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8ee03-c38e-4c40-9533-9600c679bc88 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:03.538074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 90/183] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:03 2022] GET /identity/v3/role_assignments?user.id=9f28b55f14b8488a9f23c276a094a5ad&scope.project.id=482e7368e31f415faa7b8b65d04ae50f&role.id=e7b95eb3b5ad4479b50aaa68e58591ea => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:03.539267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795c117-889e-465b-8c2c-64b6a8c7c014 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:03.539549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795c117-889e-465b-8c2c-64b6a8c7c014 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:03.539821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795c117-889e-465b-8c2c-64b6a8c7c014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:03.558987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c795c117-889e-465b-8c2c-64b6a8c7c014 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:03.560271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c795c117-889e-465b-8c2c-64b6a8c7c014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:03.615193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 94/184] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:03.616504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1623e-2c0c-4a67-80df-ae3a741b83a9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:03.665153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 91/185] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:03.665153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795c117-889e-465b-8c2c-64b6a8c7c014 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:03.796718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1623e-2c0c-4a67-80df-ae3a741b83a9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:03.817936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795c117-889e-465b-8c2c-64b6a8c7c014 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:03.831625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:03.835998 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:03.844136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6ad0f-b5ee-4619-8449-d1c1475c3318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:03.845794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6ad0f-b5ee-4619-8449-d1c1475c3318 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:03.845794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6ad0f-b5ee-4619-8449-d1c1475c3318 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:03.845794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6ad0f-b5ee-4619-8449-d1c1475c3318 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:03.846025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6ad0f-b5ee-4619-8449-d1c1475c3318 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:03.846218 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6ad0f-b5ee-4619-8449-d1c1475c3318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:03.847625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6ad0f-b5ee-4619-8449-d1c1475c3318 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:03.860455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 95/186] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:03.861588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6ad0f-b5ee-4619-8449-d1c1475c3318 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:03.872032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2daf79-7d7f-4a58-a8cb-6046e77d0b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:03.875801 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2daf79-7d7f-4a58-a8cb-6046e77d0b87 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:03.876155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2daf79-7d7f-4a58-a8cb-6046e77d0b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:03.876721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2daf79-7d7f-4a58-a8cb-6046e77d0b87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:03.878336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2daf79-7d7f-4a58-a8cb-6046e77d0b87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:03.879839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2daf79-7d7f-4a58-a8cb-6046e77d0b87 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:03.882287 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 92/187] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:03.883605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29761d9-627e-4a90-93af-a5afb7c2da08 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:03.884051 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29761d9-627e-4a90-93af-a5afb7c2da08 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:03.884337 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29761d9-627e-4a90-93af-a5afb7c2da08 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:03.886360 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 93/188] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:03.886938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29761d9-627e-4a90-93af-a5afb7c2da08 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:03.984125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6ad0f-b5ee-4619-8449-d1c1475c3318 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:04.000967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29761d9-627e-4a90-93af-a5afb7c2da08 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:04.016594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87689828-826d-415d-aa9f-94a0cd52bb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:04.017221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87689828-826d-415d-aa9f-94a0cd52bb9d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:04.017495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87689828-826d-415d-aa9f-94a0cd52bb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:04.017766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87689828-826d-415d-aa9f-94a0cd52bb9d None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:04.018876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87689828-826d-415d-aa9f-94a0cd52bb9d None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:04.020342 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87689828-826d-415d-aa9f-94a0cd52bb9d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:04.021952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 96/189] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:03 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:04.022682 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87689828-826d-415d-aa9f-94a0cd52bb9d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:04.035363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c503677-de0a-4393-8e0c-48f66aca2bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:04.035993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c503677-de0a-4393-8e0c-48f66aca2bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:04.036266 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c503677-de0a-4393-8e0c-48f66aca2bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:04.036533 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c503677-de0a-4393-8e0c-48f66aca2bc3 None admin] PATH_INFO: `/v3/users/18a7f75e30ab414e8bede97a5a332792` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:04.050932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c503677-de0a-4393-8e0c-48f66aca2bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=18a7f75e30ab414e8bede97a5a332792)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:04.052417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c503677-de0a-4393-8e0c-48f66aca2bc3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:04.054006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 94/190] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:03 2022] GET /identity/v3/users/18a7f75e30ab414e8bede97a5a332792 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:04.055224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88195311-b8b2-42b8-84dd-7ab68a27be84 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:04.055505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88195311-b8b2-42b8-84dd-7ab68a27be84 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:04.055894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88195311-b8b2-42b8-84dd-7ab68a27be84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:04.073681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-88195311-b8b2-42b8-84dd-7ab68a27be84 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:04.075451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-88195311-b8b2-42b8-84dd-7ab68a27be84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:04.084150 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:04.109356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:04.126615 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87689828-826d-415d-aa9f-94a0cd52bb9d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:04.141882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 95/191] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:04.142720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88195311-b8b2-42b8-84dd-7ab68a27be84 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:04.159276 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727227a1-1e13-4722-8521-58630963b029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:04.159892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727227a1-1e13-4722-8521-58630963b029 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:04.160169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727227a1-1e13-4722-8521-58630963b029 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:04.160440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727227a1-1e13-4722-8521-58630963b029 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:04.161090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727227a1-1e13-4722-8521-58630963b029 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:04.162519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727227a1-1e13-4722-8521-58630963b029 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:04.186330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 97/192] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:01:04 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:01:04.186962 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727227a1-1e13-4722-8521-58630963b029 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:04.260046 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88195311-b8b2-42b8-84dd-7ab68a27be84 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:04.302088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766826a-21a3-4e90-95c2-baec3bae76a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:04.302799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766826a-21a3-4e90-95c2-baec3bae76a0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:04.303106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766826a-21a3-4e90-95c2-baec3bae76a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:04.303396 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766826a-21a3-4e90-95c2-baec3bae76a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:04.315989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766826a-21a3-4e90-95c2-baec3bae76a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4766826a-21a3-4e90-95c2-baec3bae76a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:04.321555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:04.322497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 96/193] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:01:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:04.328721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766826a-21a3-4e90-95c2-baec3bae76a0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:04.343255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727227a1-1e13-4722-8521-58630963b029 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:04.362168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:04.368392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:04.386635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e3f76-eebc-41cd-8a3b-96e3532df6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:04.388283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e3f76-eebc-41cd-8a3b-96e3532df6ac None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:04.388728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e3f76-eebc-41cd-8a3b-96e3532df6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:04.389007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e3f76-eebc-41cd-8a3b-96e3532df6ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:04.389746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e3f76-eebc-41cd-8a3b-96e3532df6ac None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=482e7368e31f415faa7b8b65d04ae50f, user.id=a4f7da3ad6214b7b94cd5b55e583b917) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:04.390053 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e3f76-eebc-41cd-8a3b-96e3532df6ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:04.391543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e3f76-eebc-41cd-8a3b-96e3532df6ac None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:04.398274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 98/194] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:04 2022] GET /identity/v3/role_assignments?user.id=a4f7da3ad6214b7b94cd5b55e583b917&scope.project.id=482e7368e31f415faa7b8b65d04ae50f&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:04.399190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e3f76-eebc-41cd-8a3b-96e3532df6ac None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:04.467324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766826a-21a3-4e90-95c2-baec3bae76a0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:04.509726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e98698-4e36-4e92-a2c8-01597593d426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:04.510380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e98698-4e36-4e92-a2c8-01597593d426 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:04.510660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e98698-4e36-4e92-a2c8-01597593d426 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:04.510946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e98698-4e36-4e92-a2c8-01597593d426 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:04.537607 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e3f76-eebc-41cd-8a3b-96e3532df6ac None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:04.593246 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88accdba-52b3-4e7b-9104-9dfb7e5f6718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:04.593877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88accdba-52b3-4e7b-9104-9dfb7e5f6718 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:04.594155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88accdba-52b3-4e7b-9104-9dfb7e5f6718 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:04.594573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88accdba-52b3-4e7b-9104-9dfb7e5f6718 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:04.595668 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88accdba-52b3-4e7b-9104-9dfb7e5f6718 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:04.595848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88accdba-52b3-4e7b-9104-9dfb7e5f6718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:04.597322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88accdba-52b3-4e7b-9104-9dfb7e5f6718 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:04.608024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 99/195] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:04.608895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88accdba-52b3-4e7b-9104-9dfb7e5f6718 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:04.615306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e98698-4e36-4e92-a2c8-01597593d426 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:04.616585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e98698-4e36-4e92-a2c8-01597593d426 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:04.658680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:04.671320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:04.748962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 97/196] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:04.749816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e98698-4e36-4e92-a2c8-01597593d426 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:04.749947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88accdba-52b3-4e7b-9104-9dfb7e5f6718 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:04.788338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2da754-1a29-4016-ad4f-32161eba3f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:04.789091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2da754-1a29-4016-ad4f-32161eba3f5f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:04.789371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2da754-1a29-4016-ad4f-32161eba3f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:04.789643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2da754-1a29-4016-ad4f-32161eba3f5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:04.801539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2da754-1a29-4016-ad4f-32161eba3f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-4a2da754-1a29-4016-ad4f-32161eba3f5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:04.807350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:04.808239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:04.809071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:04.809071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:04.809071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:04.809071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:04.809071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:04.809071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:04.809071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:04.809071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:04.809071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:04.809071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:04.809071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:04.809071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:04.809071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 100/197] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:04.809986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2da754-1a29-4016-ad4f-32161eba3f5f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:04.878786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e98698-4e36-4e92-a2c8-01597593d426 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:04.927956 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:04.935662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:04.936039 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc65ac5-631b-4004-a3aa-553012a4dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:04.937426 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc65ac5-631b-4004-a3aa-553012a4dc0b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:04.938021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc65ac5-631b-4004-a3aa-553012a4dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:04.938477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc65ac5-631b-4004-a3aa-553012a4dc0b None admin] PATH_INFO: `/v3/users/18a7f75e30ab414e8bede97a5a332792` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:04.939891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc65ac5-631b-4004-a3aa-553012a4dc0b None admin] RBAC: Authorizing `identity:get_user(user_id=18a7f75e30ab414e8bede97a5a332792)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:04.940400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2da754-1a29-4016-ad4f-32161eba3f5f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:04.941661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc65ac5-631b-4004-a3aa-553012a4dc0b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:04.944416 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 98/198] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:04 2022] GET /identity/v3/users/18a7f75e30ab414e8bede97a5a332792 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:04.945412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc65ac5-631b-4004-a3aa-553012a4dc0b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:04.982123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcea563-dfca-486e-8359-624e376a899b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:04.982756 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcea563-dfca-486e-8359-624e376a899b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:04.983032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcea563-dfca-486e-8359-624e376a899b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:04.983550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcea563-dfca-486e-8359-624e376a899b None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:04.985167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcea563-dfca-486e-8359-624e376a899b None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:04.986887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcea563-dfca-486e-8359-624e376a899b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:04.988722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 101/199] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:04 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:04.989314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcea563-dfca-486e-8359-624e376a899b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:05.061440 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc65ac5-631b-4004-a3aa-553012a4dc0b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:05.100043 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee73a-5f66-4eb5-b8cd-8e665dd5fac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:05.100658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee73a-5f66-4eb5-b8cd-8e665dd5fac5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:05.100941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee73a-5f66-4eb5-b8cd-8e665dd5fac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:05.101308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee73a-5f66-4eb5-b8cd-8e665dd5fac5 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:05.102069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee73a-5f66-4eb5-b8cd-8e665dd5fac5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:05.102372 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee73a-5f66-4eb5-b8cd-8e665dd5fac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:05.103898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee73a-5f66-4eb5-b8cd-8e665dd5fac5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:05.108324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcea563-dfca-486e-8359-624e376a899b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:05.113484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 99/200] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:05.114371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee73a-5f66-4eb5-b8cd-8e665dd5fac5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:05.151278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9976e-8376-4ca5-bb40-3189745829c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:05.151968 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9976e-8376-4ca5-bb40-3189745829c4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:05.152246 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9976e-8376-4ca5-bb40-3189745829c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:05.153129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9976e-8376-4ca5-bb40-3189745829c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:05.181977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:05.199377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:05.234086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee73a-5f66-4eb5-b8cd-8e665dd5fac5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:05.240671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9976e-8376-4ca5-bb40-3189745829c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:05.242128 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9976e-8376-4ca5-bb40-3189745829c4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:05.272850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a99a5-881b-4467-9b16-f0782fc66773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:05.273855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a99a5-881b-4467-9b16-f0782fc66773 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:05.274469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a99a5-881b-4467-9b16-f0782fc66773 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:05.274934 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a99a5-881b-4467-9b16-f0782fc66773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:05.275962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a99a5-881b-4467-9b16-f0782fc66773 None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=02dd689032dc41219d425e2f5ccadc7a, user.id=18a7f75e30ab414e8bede97a5a332792) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:05.276474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a99a5-881b-4467-9b16-f0782fc66773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:05.279365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a99a5-881b-4467-9b16-f0782fc66773 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:05.285920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 100/201] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:05 2022] GET /identity/v3/role_assignments?user.id=18a7f75e30ab414e8bede97a5a332792&scope.project.id=02dd689032dc41219d425e2f5ccadc7a&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:05.287501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a99a5-881b-4467-9b16-f0782fc66773 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:05.439286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 102/202] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:05.450381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9976e-8376-4ca5-bb40-3189745829c4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:05.516166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a99a5-881b-4467-9b16-f0782fc66773 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:05.551087 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:05.564783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489bedc-b8eb-47c2-9e64-4febd33cdec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:05.566008 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:05.576518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489bedc-b8eb-47c2-9e64-4febd33cdec2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:05.576518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489bedc-b8eb-47c2-9e64-4febd33cdec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:05.576518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489bedc-b8eb-47c2-9e64-4febd33cdec2 None admin] PATH_INFO: `/v3/users/2e955f68f3a445d4b5a9329b9d2c198c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:05.576518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489bedc-b8eb-47c2-9e64-4febd33cdec2 None admin] RBAC: Authorizing `identity:get_user(user_id=2e955f68f3a445d4b5a9329b9d2c198c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:05.576518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489bedc-b8eb-47c2-9e64-4febd33cdec2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:05.576518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 101/203] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:05 2022] GET /identity/v3/users/2e955f68f3a445d4b5a9329b9d2c198c => generated 289 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:05.579883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489bedc-b8eb-47c2-9e64-4febd33cdec2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:05.626036 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9976e-8376-4ca5-bb40-3189745829c4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:05.700465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ad15d-b018-4079-887e-28ff9720856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:05.701083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ad15d-b018-4079-887e-28ff9720856a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:05.701364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ad15d-b018-4079-887e-28ff9720856a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:05.701633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ad15d-b018-4079-887e-28ff9720856a None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:05.702820 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ad15d-b018-4079-887e-28ff9720856a None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:05.704299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ad15d-b018-4079-887e-28ff9720856a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:05.707138 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 103/204] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:05 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:05.716136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ad15d-b018-4079-887e-28ff9720856a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:05.807492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489bedc-b8eb-47c2-9e64-4febd33cdec2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:05.863934 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c25f9-69e6-4301-affc-5937f82ab1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:05.864987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c25f9-69e6-4301-affc-5937f82ab1de None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:05.865625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c25f9-69e6-4301-affc-5937f82ab1de None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:05.866729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c25f9-69e6-4301-affc-5937f82ab1de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:05.870566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ad15d-b018-4079-887e-28ff9720856a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:05.941753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1035f9-3f54-4b13-98e7-6316b40f4853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:05.942754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1035f9-3f54-4b13-98e7-6316b40f4853 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:05.943258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1035f9-3f54-4b13-98e7-6316b40f4853 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:05.944642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1035f9-3f54-4b13-98e7-6316b40f4853 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:05.945860 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1035f9-3f54-4b13-98e7-6316b40f4853 None admin] RBAC: Adding query filter params (role.id=0238ab71fb17471cb87e84097272c752, scope.project.id=02dd689032dc41219d425e2f5ccadc7a, user.id=18a7f75e30ab414e8bede97a5a332792) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:05.947463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1035f9-3f54-4b13-98e7-6316b40f4853 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:05.949667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1035f9-3f54-4b13-98e7-6316b40f4853 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:05.957029 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:05.960940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:05.966817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 104/205] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:05 2022] GET /identity/v3/role_assignments?user.id=18a7f75e30ab414e8bede97a5a332792&scope.project.id=02dd689032dc41219d425e2f5ccadc7a&role.id=0238ab71fb17471cb87e84097272c752 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:05.981085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c25f9-69e6-4301-affc-5937f82ab1de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:05.982902 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c25f9-69e6-4301-affc-5937f82ab1de None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:06.134632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 102/206] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:06.140442 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1035f9-3f54-4b13-98e7-6316b40f4853 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:06.319916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1035f9-3f54-4b13-98e7-6316b40f4853 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:06.371815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc877018-011a-46d3-9dc4-f37d91f335f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:06.372528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc877018-011a-46d3-9dc4-f37d91f335f4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:06.372814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc877018-011a-46d3-9dc4-f37d91f335f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:06.373096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc877018-011a-46d3-9dc4-f37d91f335f4 None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:06.374330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc877018-011a-46d3-9dc4-f37d91f335f4 None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:06.375831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc877018-011a-46d3-9dc4-f37d91f335f4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:06.377807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 105/207] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:06 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:06.383716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c25f9-69e6-4301-affc-5937f82ab1de None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:06.512411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:06.524791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c25f9-69e6-4301-affc-5937f82ab1de None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:06.573308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c6c5c-2002-4f5a-a559-15656d4729a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:06.574794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c6c5c-2002-4f5a-a559-15656d4729a7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:06.575454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c6c5c-2002-4f5a-a559-15656d4729a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:06.576073 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c6c5c-2002-4f5a-a559-15656d4729a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:06.577357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c6c5c-2002-4f5a-a559-15656d4729a7 None admin] RBAC: Adding query filter params (role.id=fb6fc2065aaa436db8a8d1d8c7f60ade, scope.project.id=482e7368e31f415faa7b8b65d04ae50f, user.id=2e955f68f3a445d4b5a9329b9d2c198c) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:06.578042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c6c5c-2002-4f5a-a559-15656d4729a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:06.583891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c6c5c-2002-4f5a-a559-15656d4729a7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:06.591130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 103/208] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:06 2022] GET /identity/v3/role_assignments?user.id=2e955f68f3a445d4b5a9329b9d2c198c&scope.project.id=482e7368e31f415faa7b8b65d04ae50f&role.id=fb6fc2065aaa436db8a8d1d8c7f60ade => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:06.772199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ea0e3-8fd0-471b-be9e-62eabdf5234b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:06.772563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ea0e3-8fd0-471b-be9e-62eabdf5234b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:06.772751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ea0e3-8fd0-471b-be9e-62eabdf5234b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:06.774510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 106/209] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:06.780592 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b0577-dc0e-476a-97d3-221fce1a7247 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:06.781148 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b0577-dc0e-476a-97d3-221fce1a7247 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:06.781633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b0577-dc0e-476a-97d3-221fce1a7247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:06.804733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-784b0577-dc0e-476a-97d3-221fce1a7247 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:06.806278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-784b0577-dc0e-476a-97d3-221fce1a7247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:06.920882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 104/210] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:07.005051 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6cc2c-ddf6-464a-a928-6c4f86fad5b0 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:07.005405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6cc2c-ddf6-464a-a928-6c4f86fad5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:07.005610 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6cc2c-ddf6-464a-a928-6c4f86fad5b0 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:07.007130 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 107/211] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:07.012823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5708f2b-6dfd-447b-827c-676317cbb1d5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:07.013353 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5708f2b-6dfd-447b-827c-676317cbb1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:07.013849 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5708f2b-6dfd-447b-827c-676317cbb1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:07.032897 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a5708f2b-6dfd-447b-827c-676317cbb1d5 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:07.034390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a5708f2b-6dfd-447b-827c-676317cbb1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:07.107822 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 105/212] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:07.234699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6cc2c-ddf6-464a-a928-6c4f86fad5b0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:07.319304 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:07.364098 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6cc2c-ddf6-464a-a928-6c4f86fad5b0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:07.393619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5708f2b-6dfd-447b-827c-676317cbb1d5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:07.404712 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8418ff-4d12-4521-bf59-d3a420cb8d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:07.405354 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8418ff-4d12-4521-bf59-d3a420cb8d62 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:07.405635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8418ff-4d12-4521-bf59-d3a420cb8d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:07.406225 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8418ff-4d12-4521-bf59-d3a420cb8d62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:07.415610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:07.423865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8418ff-4d12-4521-bf59-d3a420cb8d62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0d8418ff-4d12-4521-bf59-d3a420cb8d62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:07.430126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:07.431039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:07.431864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:07.431864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:07.431864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:07.431864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:07.431864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:07.431864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:07.431864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:07.431864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:07.431864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:07.431864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:07.431864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:07.431864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:07.434448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 108/213] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:07.440273 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8418ff-4d12-4521-bf59-d3a420cb8d62 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:07.534554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5708f2b-6dfd-447b-827c-676317cbb1d5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:07.565455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8418ff-4d12-4521-bf59-d3a420cb8d62 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:07.578040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba750cf-913d-49c4-98f7-4321a3918ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:07.578681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba750cf-913d-49c4-98f7-4321a3918ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:07.578954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba750cf-913d-49c4-98f7-4321a3918ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:07.579269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba750cf-913d-49c4-98f7-4321a3918ca3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:07.580944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba750cf-913d-49c4-98f7-4321a3918ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:07.583131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba750cf-913d-49c4-98f7-4321a3918ca3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:07.585680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 106/214] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:07.590738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba750cf-913d-49c4-98f7-4321a3918ca3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:07.606176 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:07.620124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb9268-6b14-47ea-98c4-0c0c45e2a3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:07.621711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb9268-6b14-47ea-98c4-0c0c45e2a3ee None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:07.622167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb9268-6b14-47ea-98c4-0c0c45e2a3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:07.622563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb9268-6b14-47ea-98c4-0c0c45e2a3ee None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:07.623464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb9268-6b14-47ea-98c4-0c0c45e2a3ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:07.623876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb9268-6b14-47ea-98c4-0c0c45e2a3ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:07.625353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb9268-6b14-47ea-98c4-0c0c45e2a3ee None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:07.634910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 109/215] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:07.641008 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb9268-6b14-47ea-98c4-0c0c45e2a3ee None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:07.682987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:07.699010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba750cf-913d-49c4-98f7-4321a3918ca3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:07.735320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c19fc4-a3ed-4984-8ecf-5a78812e07ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:07.736305 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c19fc4-a3ed-4984-8ecf-5a78812e07ab None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:07.736785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c19fc4-a3ed-4984-8ecf-5a78812e07ab None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:07.737246 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c19fc4-a3ed-4984-8ecf-5a78812e07ab None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:07.738079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c19fc4-a3ed-4984-8ecf-5a78812e07ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:07.739738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c19fc4-a3ed-4984-8ecf-5a78812e07ab None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:07.743118 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb9268-6b14-47ea-98c4-0c0c45e2a3ee None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:07.772889 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 107/216] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:01:07 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:01:07.791926 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fbd21-9752-49ba-abf2-4bbfd4dac947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:07.792507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fbd21-9752-49ba-abf2-4bbfd4dac947 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:07.792965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fbd21-9752-49ba-abf2-4bbfd4dac947 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:07.793258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fbd21-9752-49ba-abf2-4bbfd4dac947 None admin] PATH_INFO: `/v3/users/57c677087d104d879d42eae0473a5e79` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:07.811366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fbd21-9752-49ba-abf2-4bbfd4dac947 None admin] RBAC: Authorizing `identity:get_user(user_id=57c677087d104d879d42eae0473a5e79)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:07.813172 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fbd21-9752-49ba-abf2-4bbfd4dac947 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:07.815597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 110/217] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:07 2022] GET /identity/v3/users/57c677087d104d879d42eae0473a5e79 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:07.821579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c19fc4-a3ed-4984-8ecf-5a78812e07ab None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:07.875479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c6d4b-5536-431d-a9e8-b61cdb75d4eb None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:07.875944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c6d4b-5536-431d-a9e8-b61cdb75d4eb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:07.876111 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c6d4b-5536-431d-a9e8-b61cdb75d4eb None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:07.877673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 111/218] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:07.883219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869802a-4762-46ac-b072-bc54513bc609 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:07.883517 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869802a-4762-46ac-b072-bc54513bc609 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:07.883770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869802a-4762-46ac-b072-bc54513bc609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:07.904966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8869802a-4762-46ac-b072-bc54513bc609 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:07.906676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8869802a-4762-46ac-b072-bc54513bc609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:07.942608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c19fc4-a3ed-4984-8ecf-5a78812e07ab None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:07.959390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:07.983339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807b0e8-14ed-449c-8310-eefeabba3a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:07.984000 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807b0e8-14ed-449c-8310-eefeabba3a44 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:07.984278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807b0e8-14ed-449c-8310-eefeabba3a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:07.984548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807b0e8-14ed-449c-8310-eefeabba3a44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:07.987852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 112/219] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:01:07.998913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:08.060873 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807b0e8-14ed-449c-8310-eefeabba3a44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:08.062721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807b0e8-14ed-449c-8310-eefeabba3a44 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:08.173178 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 108/220] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:08.178611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869802a-4762-46ac-b072-bc54513bc609 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:08.280629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807b0e8-14ed-449c-8310-eefeabba3a44 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:08.311719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869802a-4762-46ac-b072-bc54513bc609 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:08.329566 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:08.359243 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282055dd-8d42-4b09-8318-81386788cf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:08.360601 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282055dd-8d42-4b09-8318-81386788cf2b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:08.361303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282055dd-8d42-4b09-8318-81386788cf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:08.361721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282055dd-8d42-4b09-8318-81386788cf2b None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:08.363742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282055dd-8d42-4b09-8318-81386788cf2b None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:08.365644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282055dd-8d42-4b09-8318-81386788cf2b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:08.368257 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 113/221] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:08 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:08.376189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282055dd-8d42-4b09-8318-81386788cf2b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:08.421042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807b0e8-14ed-449c-8310-eefeabba3a44 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:08.442408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:08.485356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31c984-870f-4262-b960-5bfd672566b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:08.486403 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31c984-870f-4262-b960-5bfd672566b9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:08.488143 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31c984-870f-4262-b960-5bfd672566b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:08.489269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31c984-870f-4262-b960-5bfd672566b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:08.512124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31c984-870f-4262-b960-5bfd672566b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5b31c984-870f-4262-b960-5bfd672566b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:08.518538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:08.519676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:08.520684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:08.520684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:08.520684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:08.520684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:08.520684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:08.520684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:08.520684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:08.520684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:08.520684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:08.520684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:08.520684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:08.520684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:08.533727 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 109/222] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:01:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:01:08.537007 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31c984-870f-4262-b960-5bfd672566b9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:08.556598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282055dd-8d42-4b09-8318-81386788cf2b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:08.601061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff85c1-eee4-4ab3-8393-d17eff2d0489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:08.601712 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff85c1-eee4-4ab3-8393-d17eff2d0489 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:08.602157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff85c1-eee4-4ab3-8393-d17eff2d0489 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:08.602499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff85c1-eee4-4ab3-8393-d17eff2d0489 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:08.603455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff85c1-eee4-4ab3-8393-d17eff2d0489 None admin] RBAC: Adding query filter params (role.id=e7b95eb3b5ad4479b50aaa68e58591ea, scope.project.id=02dd689032dc41219d425e2f5ccadc7a, user.id=57c677087d104d879d42eae0473a5e79) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:08.603835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff85c1-eee4-4ab3-8393-d17eff2d0489 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:08.611525 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff85c1-eee4-4ab3-8393-d17eff2d0489 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:08.621313 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 114/223] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:08 2022] GET /identity/v3/role_assignments?user.id=57c677087d104d879d42eae0473a5e79&scope.project.id=02dd689032dc41219d425e2f5ccadc7a&role.id=e7b95eb3b5ad4479b50aaa68e58591ea => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:08.623812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74b1db-a885-4637-932e-75b06092765c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:08.624330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74b1db-a885-4637-932e-75b06092765c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:08.625081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74b1db-a885-4637-932e-75b06092765c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:08.626974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 115/224] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:08.634057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131c94e-a15a-4638-ad12-8eb499e0c7d0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:08.634617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131c94e-a15a-4638-ad12-8eb499e0c7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:08.635129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131c94e-a15a-4638-ad12-8eb499e0c7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:08.660460 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6131c94e-a15a-4638-ad12-8eb499e0c7d0 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:08.662437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6131c94e-a15a-4638-ad12-8eb499e0c7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:08.663462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:08.666879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31c984-870f-4262-b960-5bfd672566b9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:08.708988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d710c0-c785-4f52-add5-f2482b405469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:08.709903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d710c0-c785-4f52-add5-f2482b405469 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:08.710354 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d710c0-c785-4f52-add5-f2482b405469 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:08.710828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d710c0-c785-4f52-add5-f2482b405469 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:08.711708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d710c0-c785-4f52-add5-f2482b405469 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:08.712200 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d710c0-c785-4f52-add5-f2482b405469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:08.713870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d710c0-c785-4f52-add5-f2482b405469 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:08.723956 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 110/225] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:08.736234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d710c0-c785-4f52-add5-f2482b405469 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:08.751083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 116/226] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:08.887705 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d710c0-c785-4f52-add5-f2482b405469 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:08.927413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74845aa0-41ed-428d-b956-ca3c64c6bd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:08.928351 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74845aa0-41ed-428d-b956-ca3c64c6bd09 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:08.928837 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74845aa0-41ed-428d-b956-ca3c64c6bd09 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:08.929283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74845aa0-41ed-428d-b956-ca3c64c6bd09 None admin] PATH_INFO: `/v3/users/18a7f75e30ab414e8bede97a5a332792` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:08.930716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74845aa0-41ed-428d-b956-ca3c64c6bd09 None admin] RBAC: Authorizing `identity:get_user(user_id=18a7f75e30ab414e8bede97a5a332792)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:08.932390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74845aa0-41ed-428d-b956-ca3c64c6bd09 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:08.935085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 111/227] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:08 2022] GET /identity/v3/users/18a7f75e30ab414e8bede97a5a332792 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:08.940863 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131c94e-a15a-4638-ad12-8eb499e0c7d0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:08.989151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:09.069720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131c94e-a15a-4638-ad12-8eb499e0c7d0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:09.111090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086e6f4-cf0a-4a90-a0bd-f672a003806c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:09.111774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086e6f4-cf0a-4a90-a0bd-f672a003806c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:09.112049 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086e6f4-cf0a-4a90-a0bd-f672a003806c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:09.112326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086e6f4-cf0a-4a90-a0bd-f672a003806c None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:09.113585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086e6f4-cf0a-4a90-a0bd-f672a003806c None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:09.114419 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74845aa0-41ed-428d-b956-ca3c64c6bd09 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:09.115096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086e6f4-cf0a-4a90-a0bd-f672a003806c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:09.117256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 117/228] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:08 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:09.124027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086e6f4-cf0a-4a90-a0bd-f672a003806c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:09.171750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:09.282541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74845aa0-41ed-428d-b956-ca3c64c6bd09 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:09.325302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c6b1d-f2bc-4eec-a571-ce3b0314012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:09.326487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c6b1d-f2bc-4eec-a571-ce3b0314012b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:09.326990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c6b1d-f2bc-4eec-a571-ce3b0314012b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:09.327475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c6b1d-f2bc-4eec-a571-ce3b0314012b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:09.334629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086e6f4-cf0a-4a90-a0bd-f672a003806c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:09.350355 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c6b1d-f2bc-4eec-a571-ce3b0314012b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-585c6b1d-f2bc-4eec-a571-ce3b0314012b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:09.356389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:09.357267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:09.358065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:09.358065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:09.358065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:09.358065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:09.358065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:09.358065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:09.358065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:09.358065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:09.358065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:09.358065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:09.358065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:09.358065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:09.361382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 112/229] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:01:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:09.363557 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:09.376133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c6b1d-f2bc-4eec-a571-ce3b0314012b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:09.402802 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe3634-10e3-4e61-8a6e-715a0ef489cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:09.403513 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe3634-10e3-4e61-8a6e-715a0ef489cd None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:09.403794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe3634-10e3-4e61-8a6e-715a0ef489cd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:09.404071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe3634-10e3-4e61-8a6e-715a0ef489cd None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a/users/18a7f75e30ab414e8bede97a5a332792/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:09.407548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe3634-10e3-4e61-8a6e-715a0ef489cd None admin] RBAC: Authorizing `identity:create_grant(project_id=02dd689032dc41219d425e2f5ccadc7a, user_id=18a7f75e30ab414e8bede97a5a332792, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:09.409010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe3634-10e3-4e61-8a6e-715a0ef489cd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:09.431629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 118/230] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:01:09 2022] PUT /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a/users/18a7f75e30ab414e8bede97a5a332792/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:01:09.512340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:09.525117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c6b1d-f2bc-4eec-a571-ce3b0314012b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:09.579346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d1e47-4490-480b-98fa-39ba198d512a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:09.580098 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d1e47-4490-480b-98fa-39ba198d512a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:09.580360 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d1e47-4490-480b-98fa-39ba198d512a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:09.580643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d1e47-4490-480b-98fa-39ba198d512a None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:09.581387 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d1e47-4490-480b-98fa-39ba198d512a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:09.581702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d1e47-4490-480b-98fa-39ba198d512a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:09.583146 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d1e47-4490-480b-98fa-39ba198d512a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:09.598960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 113/231] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:09.604338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe3634-10e3-4e61-8a6e-715a0ef489cd None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:09.742055 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe3634-10e3-4e61-8a6e-715a0ef489cd None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:09.782828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b9de0-7286-4423-90ce-fa48cf61be9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:09.783533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b9de0-7286-4423-90ce-fa48cf61be9e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:09.783846 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b9de0-7286-4423-90ce-fa48cf61be9e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:09.784136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b9de0-7286-4423-90ce-fa48cf61be9e None admin] PATH_INFO: `/v3/users/18a7f75e30ab414e8bede97a5a332792` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:09.785391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b9de0-7286-4423-90ce-fa48cf61be9e None admin] RBAC: Authorizing `identity:get_user(user_id=18a7f75e30ab414e8bede97a5a332792)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:09.786903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b9de0-7286-4423-90ce-fa48cf61be9e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:09.788609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 119/232] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:09 2022] GET /identity/v3/users/18a7f75e30ab414e8bede97a5a332792 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:09.794420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d1e47-4490-480b-98fa-39ba198d512a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:09.922195 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d1e47-4490-480b-98fa-39ba198d512a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:09.972757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653d071-5eae-459b-b359-41b8f08be4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:09.973407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653d071-5eae-459b-b359-41b8f08be4ae None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:09.973746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653d071-5eae-459b-b359-41b8f08be4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:09.974025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653d071-5eae-459b-b359-41b8f08be4ae None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:09.975200 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653d071-5eae-459b-b359-41b8f08be4ae None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:09.976660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653d071-5eae-459b-b359-41b8f08be4ae None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:09.978373 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 114/233] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:09 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:09.984255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b9de0-7286-4423-90ce-fa48cf61be9e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:10.025669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:10.110511 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b9de0-7286-4423-90ce-fa48cf61be9e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:10.154752 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0511f-d4a6-4d8c-a115-1b022643022e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:10.155459 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0511f-d4a6-4d8c-a115-1b022643022e None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:10.155741 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0511f-d4a6-4d8c-a115-1b022643022e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:10.156013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0511f-d4a6-4d8c-a115-1b022643022e None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a/users/18a7f75e30ab414e8bede97a5a332792/roles/0238ab71fb17471cb87e84097272c752` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:10.158261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0511f-d4a6-4d8c-a115-1b022643022e None admin] RBAC: Authorizing `identity:create_grant(project_id=02dd689032dc41219d425e2f5ccadc7a, user_id=18a7f75e30ab414e8bede97a5a332792, role_id=0238ab71fb17471cb87e84097272c752)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:10.159747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0511f-d4a6-4d8c-a115-1b022643022e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:10.180665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 120/234] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:01:09 2022] PUT /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a/users/18a7f75e30ab414e8bede97a5a332792/roles/0238ab71fb17471cb87e84097272c752 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:01:10.617530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a37086-508d-4fb2-a409-e838e226da93 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:10.618004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a37086-508d-4fb2-a409-e838e226da93 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:10.618076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a37086-508d-4fb2-a409-e838e226da93 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:10.619650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 115/235] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:10.625436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae4fd8-b897-4526-86ce-c3201efc3fef None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:10.625875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae4fd8-b897-4526-86ce-c3201efc3fef None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:10.626062 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae4fd8-b897-4526-86ce-c3201efc3fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:10.652372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b4ae4fd8-b897-4526-86ce-c3201efc3fef None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:10.653622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b4ae4fd8-b897-4526-86ce-c3201efc3fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:10.741705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 121/236] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:10.881077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18382d-6099-4490-b9b7-02025ab61fc4 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:10.881369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18382d-6099-4490-b9b7-02025ab61fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:10.881549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18382d-6099-4490-b9b7-02025ab61fc4 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:10.882911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 116/237] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:10.888830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e07116-6d5c-4a0c-8f9d-25027a40eb51 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:10.889124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e07116-6d5c-4a0c-8f9d-25027a40eb51 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:10.889405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e07116-6d5c-4a0c-8f9d-25027a40eb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:10.908396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-31e07116-6d5c-4a0c-8f9d-25027a40eb51 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:10.909660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-31e07116-6d5c-4a0c-8f9d-25027a40eb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:10.932779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:10.998968 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 122/238] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:11.110947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18382d-6099-4490-b9b7-02025ab61fc4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:11.135569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:11.244936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18382d-6099-4490-b9b7-02025ab61fc4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:11.294366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd0e2a-e19f-4038-8049-57358ecc96f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:11.295021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd0e2a-e19f-4038-8049-57358ecc96f2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:11.295330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd0e2a-e19f-4038-8049-57358ecc96f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:11.295602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd0e2a-e19f-4038-8049-57358ecc96f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:11.306438 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd0e2a-e19f-4038-8049-57358ecc96f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-dbbd0e2a-e19f-4038-8049-57358ecc96f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:11.312733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:11.313603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:11.314673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:11.314673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:11.314673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:11.314673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:11.314673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:11.314673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:11.314673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:11.314673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:11.314673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:11.314673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:11.314673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:11.314673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:11.314673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 117/239] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:11.319988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e07116-6d5c-4a0c-8f9d-25027a40eb51 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:11.387208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767dfc1-8af9-44ee-a30c-575749fcef6c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:11.387405 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767dfc1-8af9-44ee-a30c-575749fcef6c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:11.387651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767dfc1-8af9-44ee-a30c-575749fcef6c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:11.389006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 118/240] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:11.399372 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abade88d-4468-439b-8d05-603521e87a2a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:11.399651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abade88d-4468-439b-8d05-603521e87a2a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:11.399927 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abade88d-4468-439b-8d05-603521e87a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:11.426038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-abade88d-4468-439b-8d05-603521e87a2a None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:11.427285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-abade88d-4468-439b-8d05-603521e87a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:11.459990 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e07116-6d5c-4a0c-8f9d-25027a40eb51 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:11.499140 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c002cc7-79cf-4b8e-8c42-35d0b43ab9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:11.500106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c002cc7-79cf-4b8e-8c42-35d0b43ab9fa None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:11.500648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c002cc7-79cf-4b8e-8c42-35d0b43ab9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:11.501096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c002cc7-79cf-4b8e-8c42-35d0b43ab9fa None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:11.501907 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c002cc7-79cf-4b8e-8c42-35d0b43ab9fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:11.502354 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c002cc7-79cf-4b8e-8c42-35d0b43ab9fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:11.504192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c002cc7-79cf-4b8e-8c42-35d0b43ab9fa None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:11.509130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 119/241] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:11.510564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abade88d-4468-439b-8d05-603521e87a2a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:11.513976 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 123/242] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:11.524050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c002cc7-79cf-4b8e-8c42-35d0b43ab9fa None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:11.527388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:11.547248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:11.646324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c002cc7-79cf-4b8e-8c42-35d0b43ab9fa None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:11.684746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a2818-541c-4d4c-b6d2-5e076101177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:11.687635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a2818-541c-4d4c-b6d2-5e076101177e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:11.689189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a2818-541c-4d4c-b6d2-5e076101177e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:11.689626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a2818-541c-4d4c-b6d2-5e076101177e None admin] PATH_INFO: `/v3/users/27d14ad794d24c01b7a424f22564a745` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:11.719187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a2818-541c-4d4c-b6d2-5e076101177e None admin] RBAC: Authorizing `identity:get_user(user_id=27d14ad794d24c01b7a424f22564a745)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:11.721461 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a2818-541c-4d4c-b6d2-5e076101177e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:11.723479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 124/243] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:11 2022] GET /identity/v3/users/27d14ad794d24c01b7a424f22564a745 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:11.731195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a2818-541c-4d4c-b6d2-5e076101177e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:11.736420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abade88d-4468-439b-8d05-603521e87a2a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:11.808232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7430804-f72d-4753-a93b-8f8910bebf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:11.809117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7430804-f72d-4753-a93b-8f8910bebf7d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:11.809538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7430804-f72d-4753-a93b-8f8910bebf7d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:11.809951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7430804-f72d-4753-a93b-8f8910bebf7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:11.811291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7430804-f72d-4753-a93b-8f8910bebf7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:11.815159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7430804-f72d-4753-a93b-8f8910bebf7d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:11.817424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 120/244] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:11.823665 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7430804-f72d-4753-a93b-8f8910bebf7d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:11.879957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:11.912762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a2818-541c-4d4c-b6d2-5e076101177e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:11.941207 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:11.966013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7430804-f72d-4753-a93b-8f8910bebf7d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:11.966539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d2a2c-87c2-4328-8693-af416f27d63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:11.967185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d2a2c-87c2-4328-8693-af416f27d63f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:11.967470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d2a2c-87c2-4328-8693-af416f27d63f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:11.967745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d2a2c-87c2-4328-8693-af416f27d63f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:12.003847 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6aa10-c1c3-432c-bc0c-51d370ddb0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:12.004898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6aa10-c1c3-432c-bc0c-51d370ddb0cf None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:12.005347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6aa10-c1c3-432c-bc0c-51d370ddb0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:12.005830 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6aa10-c1c3-432c-bc0c-51d370ddb0cf None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:12.006688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6aa10-c1c3-432c-bc0c-51d370ddb0cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:12.008406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6aa10-c1c3-432c-bc0c-51d370ddb0cf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:12.036718 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 121/245] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:01:11 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:01:12.038079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6aa10-c1c3-432c-bc0c-51d370ddb0cf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:12.080733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d2a2c-87c2-4328-8693-af416f27d63f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:12.084323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d2a2c-87c2-4328-8693-af416f27d63f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:12.229775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6aa10-c1c3-432c-bc0c-51d370ddb0cf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:12.232722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:12.265045 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 125/246] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:12.266650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134c9d7-a186-4091-bdf2-a22142eac4f6 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:12.267175 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134c9d7-a186-4091-bdf2-a22142eac4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:12.267653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134c9d7-a186-4091-bdf2-a22142eac4f6 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:12.269187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 126/247] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:12.275332 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134c9d7-a186-4091-bdf2-a22142eac4f6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:12.279598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd0d71-8013-4111-8387-d77c8a7636e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:12.280518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd0d71-8013-4111-8387-d77c8a7636e7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:12.281018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd0d71-8013-4111-8387-d77c8a7636e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:12.281486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd0d71-8013-4111-8387-d77c8a7636e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:12.283935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:12.296351 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd0d71-8013-4111-8387-d77c8a7636e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-25fd0d71-8013-4111-8387-d77c8a7636e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:12.301317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:12.302238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:12.303093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:12.303093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:12.303093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:12.303093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:12.303093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:12.303093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:12.303093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:12.303093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:12.303093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:12.303093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:12.303093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:12.303093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:12.303093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 122/248] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:12.304454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9c768-2a58-4a77-b139-429a81a61dba None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:12.304454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9c768-2a58-4a77-b139-429a81a61dba None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:12.304454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9c768-2a58-4a77-b139-429a81a61dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:12.335135 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5dc9c768-2a58-4a77-b139-429a81a61dba None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:12.336419 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5dc9c768-2a58-4a77-b139-429a81a61dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:12.414425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 123/249] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:12.415538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9c768-2a58-4a77-b139-429a81a61dba None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:12.420126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134c9d7-a186-4091-bdf2-a22142eac4f6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:12.459628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68cdbb-3286-45f5-b7b8-a5f3556a9a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:12.460303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68cdbb-3286-45f5-b7b8-a5f3556a9a4b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:12.460584 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68cdbb-3286-45f5-b7b8-a5f3556a9a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:12.460859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68cdbb-3286-45f5-b7b8-a5f3556a9a4b None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:12.462080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68cdbb-3286-45f5-b7b8-a5f3556a9a4b None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:12.463796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68cdbb-3286-45f5-b7b8-a5f3556a9a4b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:12.465765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 127/250] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:12 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:12.471636 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68cdbb-3286-45f5-b7b8-a5f3556a9a4b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:12.524090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:12.536303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9c768-2a58-4a77-b139-429a81a61dba None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:12.566111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:12.580333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d4e37-b097-4dfd-9454-50d5c04d4a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:12.580957 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d4e37-b097-4dfd-9454-50d5c04d4a59 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:12.581297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d4e37-b097-4dfd-9454-50d5c04d4a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:12.581502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d4e37-b097-4dfd-9454-50d5c04d4a59 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:12.582176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d4e37-b097-4dfd-9454-50d5c04d4a59 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:12.582480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d4e37-b097-4dfd-9454-50d5c04d4a59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:12.584313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d4e37-b097-4dfd-9454-50d5c04d4a59 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:12.593532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68cdbb-3286-45f5-b7b8-a5f3556a9a4b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:12.602194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 124/251] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:12.614468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d4e37-b097-4dfd-9454-50d5c04d4a59 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:12.637428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4a8a6-191d-4456-bed4-b9ffeda89460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:12.638058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4a8a6-191d-4456-bed4-b9ffeda89460 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:12.638338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4a8a6-191d-4456-bed4-b9ffeda89460 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:12.638612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4a8a6-191d-4456-bed4-b9ffeda89460 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:12.639611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4a8a6-191d-4456-bed4-b9ffeda89460 None admin] RBAC: Adding query filter params (role.id=fb6fc2065aaa436db8a8d1d8c7f60ade, scope.project.id=02dd689032dc41219d425e2f5ccadc7a, user.id=27d14ad794d24c01b7a424f22564a745) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:12.639927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4a8a6-191d-4456-bed4-b9ffeda89460 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:12.641501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4a8a6-191d-4456-bed4-b9ffeda89460 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:12.649212 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 128/252] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:12 2022] GET /identity/v3/role_assignments?user.id=27d14ad794d24c01b7a424f22564a745&scope.project.id=02dd689032dc41219d425e2f5ccadc7a&role.id=fb6fc2065aaa436db8a8d1d8c7f60ade => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:12.754754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d4e37-b097-4dfd-9454-50d5c04d4a59 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:12.790186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4a8a6-191d-4456-bed4-b9ffeda89460 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:12.801844 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606ab6f-f710-4a7a-a1db-3edae63654d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:12.802497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606ab6f-f710-4a7a-a1db-3edae63654d8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:12.802778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606ab6f-f710-4a7a-a1db-3edae63654d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:12.803070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606ab6f-f710-4a7a-a1db-3edae63654d8 None admin] PATH_INFO: `/v3/users/57c677087d104d879d42eae0473a5e79` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:12.804230 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606ab6f-f710-4a7a-a1db-3edae63654d8 None admin] RBAC: Authorizing `identity:get_user(user_id=57c677087d104d879d42eae0473a5e79)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:12.805706 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606ab6f-f710-4a7a-a1db-3edae63654d8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:12.808570 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 125/253] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:12 2022] GET /identity/v3/users/57c677087d104d879d42eae0473a5e79 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:12.815450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606ab6f-f710-4a7a-a1db-3edae63654d8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:12.942357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:12.982037 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4a8a6-191d-4456-bed4-b9ffeda89460 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:12.995703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:13.035319 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606ab6f-f710-4a7a-a1db-3edae63654d8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:13.050708 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81676d7a-7ddb-4892-8796-e6d450baa7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:13.051322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81676d7a-7ddb-4892-8796-e6d450baa7da None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:13.051600 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81676d7a-7ddb-4892-8796-e6d450baa7da None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:13.051870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81676d7a-7ddb-4892-8796-e6d450baa7da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:13.065276 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81676d7a-7ddb-4892-8796-e6d450baa7da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-81676d7a-7ddb-4892-8796-e6d450baa7da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:13.071638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:13.072545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:13.073410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:13.073410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:13.073410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:13.073410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:13.073410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:13.073410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:13.073410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:13.073410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:13.073410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:13.073410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:13.073410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:13.073410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:13.073410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 129/254] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:01:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:01:13.079155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fcbdb-7a58-4441-960a-aecf71dee87e None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:13.079155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fcbdb-7a58-4441-960a-aecf71dee87e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:13.079155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fcbdb-7a58-4441-960a-aecf71dee87e None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:13.080984 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 130/255] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:13.082047 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fcbdb-7a58-4441-960a-aecf71dee87e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:13.089404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40905e8a-0072-4d13-a267-69f935b0572c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:13.090369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40905e8a-0072-4d13-a267-69f935b0572c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:13.090798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40905e8a-0072-4d13-a267-69f935b0572c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:13.091217 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40905e8a-0072-4d13-a267-69f935b0572c None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:13.092568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40905e8a-0072-4d13-a267-69f935b0572c None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:13.094152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40905e8a-0072-4d13-a267-69f935b0572c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:13.096468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 126/256] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:12 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:13.097744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4328bc-f176-4425-a060-59ad5cc96dc4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:13.098192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4328bc-f176-4425-a060-59ad5cc96dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:13.098568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4328bc-f176-4425-a060-59ad5cc96dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:13.118121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ab4328bc-f176-4425-a060-59ad5cc96dc4 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:13.119628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ab4328bc-f176-4425-a060-59ad5cc96dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:13.196342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 127/257] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:13.197336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4328bc-f176-4425-a060-59ad5cc96dc4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:13.204774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fcbdb-7a58-4441-960a-aecf71dee87e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:13.274233 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353463a5-0d77-4263-837b-7655cc68b754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:13.274907 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353463a5-0d77-4263-837b-7655cc68b754 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:13.275215 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353463a5-0d77-4263-837b-7655cc68b754 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:13.275489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353463a5-0d77-4263-837b-7655cc68b754 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:13.276321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353463a5-0d77-4263-837b-7655cc68b754 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:13.276628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353463a5-0d77-4263-837b-7655cc68b754 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:13.278013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353463a5-0d77-4263-837b-7655cc68b754 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:13.295412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:13.298732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 131/258] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:13.312069 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353463a5-0d77-4263-837b-7655cc68b754 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:13.375818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:13.397772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4328bc-f176-4425-a060-59ad5cc96dc4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:13.458549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491c3c9-dbf8-436a-87bb-09f35348004a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:13.459254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491c3c9-dbf8-436a-87bb-09f35348004a None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:13.459532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491c3c9-dbf8-436a-87bb-09f35348004a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:13.459810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491c3c9-dbf8-436a-87bb-09f35348004a None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a/users/57c677087d104d879d42eae0473a5e79/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:13.461712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491c3c9-dbf8-436a-87bb-09f35348004a None admin] RBAC: Authorizing `identity:create_grant(project_id=02dd689032dc41219d425e2f5ccadc7a, user_id=57c677087d104d879d42eae0473a5e79, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:13.463204 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491c3c9-dbf8-436a-87bb-09f35348004a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:13.494003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 128/259] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:01:13 2022] PUT /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a/users/57c677087d104d879d42eae0473a5e79/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:01:13.521995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353463a5-0d77-4263-837b-7655cc68b754 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:13.577112 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6b505-02ee-48ed-a17d-9c7fb6b68b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:13.577687 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6b505-02ee-48ed-a17d-9c7fb6b68b45 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:13.577961 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6b505-02ee-48ed-a17d-9c7fb6b68b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:13.578227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6b505-02ee-48ed-a17d-9c7fb6b68b45 None admin] PATH_INFO: `/v3/users/18a7f75e30ab414e8bede97a5a332792` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:13.579421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6b505-02ee-48ed-a17d-9c7fb6b68b45 None admin] RBAC: Authorizing `identity:get_user(user_id=18a7f75e30ab414e8bede97a5a332792)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:13.580909 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6b505-02ee-48ed-a17d-9c7fb6b68b45 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:13.583006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 132/260] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:13 2022] GET /identity/v3/users/18a7f75e30ab414e8bede97a5a332792 => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:13.590720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491c3c9-dbf8-436a-87bb-09f35348004a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:13.628134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6b505-02ee-48ed-a17d-9c7fb6b68b45 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:13.857529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491c3c9-dbf8-436a-87bb-09f35348004a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:13.875433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:13.903738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4090d-4414-4549-a317-ae6e5ce3f53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:13.904662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4090d-4414-4549-a317-ae6e5ce3f53f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:13.905116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4090d-4414-4549-a317-ae6e5ce3f53f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:13.905563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4090d-4414-4549-a317-ae6e5ce3f53f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:13.906387 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:13.915256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6b505-02ee-48ed-a17d-9c7fb6b68b45 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:13.994931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4866a6e-90a8-4631-84ab-d88fc1a9a70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:13.995895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866a6e-90a8-4631-84ab-d88fc1a9a70b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:13.997074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866a6e-90a8-4631-84ab-d88fc1a9a70b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:13.997689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866a6e-90a8-4631-84ab-d88fc1a9a70b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:14.021617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4866a6e-90a8-4631-84ab-d88fc1a9a70b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-f4866a6e-90a8-4631-84ab-d88fc1a9a70b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:14.029103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:14.030022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:14.030805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:14.030805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:14.030805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:14.030805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:14.030805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:14.030805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:14.030805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:14.030805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:14.030805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:14.030805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:14.030805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:14.030805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:14.037278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 133/261] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:01:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:14.045047 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4090d-4414-4549-a317-ae6e5ce3f53f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:14.046636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4090d-4414-4549-a317-ae6e5ce3f53f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:14.081623 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4866a6e-90a8-4631-84ab-d88fc1a9a70b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:14.248655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 129/262] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:14.255926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4090d-4414-4549-a317-ae6e5ce3f53f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:14.277484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4866a6e-90a8-4631-84ab-d88fc1a9a70b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:14.278123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:14.374362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500520d8-c428-4030-8303-6031a881c9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:14.375512 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500520d8-c428-4030-8303-6031a881c9fd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:14.376093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500520d8-c428-4030-8303-6031a881c9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:14.376560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500520d8-c428-4030-8303-6031a881c9fd None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:14.377495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500520d8-c428-4030-8303-6031a881c9fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:14.378099 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500520d8-c428-4030-8303-6031a881c9fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:14.379731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500520d8-c428-4030-8303-6031a881c9fd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:14.399086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 134/263] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:14.407237 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500520d8-c428-4030-8303-6031a881c9fd None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:14.443231 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:14.451697 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4090d-4414-4549-a317-ae6e5ce3f53f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:14.556336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736db621-183b-4e35-aafe-121a7a5831da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:14.557851 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736db621-183b-4e35-aafe-121a7a5831da None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:14.558335 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736db621-183b-4e35-aafe-121a7a5831da None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:14.558807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736db621-183b-4e35-aafe-121a7a5831da None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:14.564259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736db621-183b-4e35-aafe-121a7a5831da None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:14.567192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736db621-183b-4e35-aafe-121a7a5831da None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:14.570550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 130/264] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:14 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:14.577247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736db621-183b-4e35-aafe-121a7a5831da None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:14.623778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500520d8-c428-4030-8303-6031a881c9fd None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:14.700848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e371d37-8ce6-44b0-a3f0-d7e85dadbdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:14.701506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e371d37-8ce6-44b0-a3f0-d7e85dadbdb6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:14.701957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e371d37-8ce6-44b0-a3f0-d7e85dadbdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:14.702149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e371d37-8ce6-44b0-a3f0-d7e85dadbdb6 None admin] PATH_INFO: `/v3/users/18a7f75e30ab414e8bede97a5a332792` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:14.704931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e371d37-8ce6-44b0-a3f0-d7e85dadbdb6 None admin] RBAC: Authorizing `identity:get_user(user_id=18a7f75e30ab414e8bede97a5a332792)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:14.706501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e371d37-8ce6-44b0-a3f0-d7e85dadbdb6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:14.708307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 135/265] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:14 2022] GET /identity/v3/users/18a7f75e30ab414e8bede97a5a332792 => generated 283 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:14.721655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e371d37-8ce6-44b0-a3f0-d7e85dadbdb6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:14.745682 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:14.783141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736db621-183b-4e35-aafe-121a7a5831da None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:14.848711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0a064-78c7-47c3-b484-4867e7b5d3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:14.849251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0a064-78c7-47c3-b484-4867e7b5d3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:14.849678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0a064-78c7-47c3-b484-4867e7b5d3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:14.849960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0a064-78c7-47c3-b484-4867e7b5d3f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:14.850695 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0a064-78c7-47c3-b484-4867e7b5d3f0 None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=02dd689032dc41219d425e2f5ccadc7a, user.id=18a7f75e30ab414e8bede97a5a332792) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:14.851006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0a064-78c7-47c3-b484-4867e7b5d3f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:14.852625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0a064-78c7-47c3-b484-4867e7b5d3f0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:14.870667 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 131/266] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:14 2022] GET /identity/v3/role_assignments?user.id=18a7f75e30ab414e8bede97a5a332792&scope.project.id=02dd689032dc41219d425e2f5ccadc7a&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:14.907415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:14.931222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e371d37-8ce6-44b0-a3f0-d7e85dadbdb6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:14.980078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ade20f-4e21-4b12-9169-0794d5924571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:14.980713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ade20f-4e21-4b12-9169-0794d5924571 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:14.980993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ade20f-4e21-4b12-9169-0794d5924571 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:14.981268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ade20f-4e21-4b12-9169-0794d5924571 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:15.078961 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ade20f-4e21-4b12-9169-0794d5924571 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:15.080508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ade20f-4e21-4b12-9169-0794d5924571 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:15.187948 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:15.202667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 136/267] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:15.208035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0a064-78c7-47c3-b484-4867e7b5d3f0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:15.381887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0a064-78c7-47c3-b484-4867e7b5d3f0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:15.427587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74344a04-4423-43a6-94bd-1486c348572b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:15.428235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74344a04-4423-43a6-94bd-1486c348572b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:15.428513 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74344a04-4423-43a6-94bd-1486c348572b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:15.428844 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74344a04-4423-43a6-94bd-1486c348572b None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:15.429991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74344a04-4423-43a6-94bd-1486c348572b None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:15.431583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74344a04-4423-43a6-94bd-1486c348572b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:15.433946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 132/268] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:15 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:15.439905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ade20f-4e21-4b12-9169-0794d5924571 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:15.477326 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34979d2a-022a-4812-af7e-01cc675e5903 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:15.477631 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34979d2a-022a-4812-af7e-01cc675e5903 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:15.477878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34979d2a-022a-4812-af7e-01cc675e5903 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:15.480369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 133/269] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:15.487636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa96db8-3c4b-489a-b7a6-342f98212c96 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:15.487915 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa96db8-3c4b-489a-b7a6-342f98212c96 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:15.488189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa96db8-3c4b-489a-b7a6-342f98212c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:15.521889 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3fa96db8-3c4b-489a-b7a6-342f98212c96 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:15.523116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3fa96db8-3c4b-489a-b7a6-342f98212c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:15.536612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:15.613040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 134/270] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:01:15.634094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ade20f-4e21-4b12-9169-0794d5924571 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:15.686217 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a74aa-6bd8-4344-b570-efd471edec87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:15.686839 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a74aa-6bd8-4344-b570-efd471edec87 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:15.687146 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a74aa-6bd8-4344-b570-efd471edec87 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:15.687425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a74aa-6bd8-4344-b570-efd471edec87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:15.688170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a74aa-6bd8-4344-b570-efd471edec87 None admin] RBAC: Adding query filter params (role.id=0238ab71fb17471cb87e84097272c752, scope.project.id=02dd689032dc41219d425e2f5ccadc7a, user.id=18a7f75e30ab414e8bede97a5a332792) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:15.688481 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a74aa-6bd8-4344-b570-efd471edec87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:15.689995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a74aa-6bd8-4344-b570-efd471edec87 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:15.696637 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 137/271] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:15 2022] GET /identity/v3/role_assignments?user.id=18a7f75e30ab414e8bede97a5a332792&scope.project.id=02dd689032dc41219d425e2f5ccadc7a&role.id=0238ab71fb17471cb87e84097272c752 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:15.917163 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa96db8-3c4b-489a-b7a6-342f98212c96 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:16.009521 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5778b26-5e97-47a3-a02c-e23566b5887f None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.011084 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5778b26-5e97-47a3-a02c-e23566b5887f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.011510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5778b26-5e97-47a3-a02c-e23566b5887f None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:16.013232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 138/272] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:16.018659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e76e9c-535b-4c84-9b98-6fd531852088 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.019144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e76e9c-535b-4c84-9b98-6fd531852088 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.019585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e76e9c-535b-4c84-9b98-6fd531852088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:16.037008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa96db8-3c4b-489a-b7a6-342f98212c96 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:16.046608 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c4e76e9c-535b-4c84-9b98-6fd531852088 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:16.048346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c4e76e9c-535b-4c84-9b98-6fd531852088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:16.079458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527977e-9bcd-4508-9ee2-8020b46eb56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:16.080102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527977e-9bcd-4508-9ee2-8020b46eb56c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.080378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527977e-9bcd-4508-9ee2-8020b46eb56c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.080759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527977e-9bcd-4508-9ee2-8020b46eb56c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:16.093568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527977e-9bcd-4508-9ee2-8020b46eb56c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e527977e-9bcd-4508-9ee2-8020b46eb56c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:16.098679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:16.099572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:16.100358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:16.100358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:16.100358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:16.100358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:16.100358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:16.100358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:16.100358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:16.100358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:16.100358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:16.100358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:16.100358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:16.100358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:16.102969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 135/273] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:16.108012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:16.110602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527977e-9bcd-4508-9ee2-8020b46eb56c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:16.114047 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:16.135311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 139/274] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:01:16.137343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd36a2-b6c2-4906-a9dd-d30cbc4d5c61 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.137629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd36a2-b6c2-4906-a9dd-d30cbc4d5c61 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.137905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd36a2-b6c2-4906-a9dd-d30cbc4d5c61 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:16.143632 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 140/275] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:16.150340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0577ab6-6797-4bb9-beb5-1950dc835fb2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.151254 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0577ab6-6797-4bb9-beb5-1950dc835fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.151515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0577ab6-6797-4bb9-beb5-1950dc835fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:16.174863 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b0577ab6-6797-4bb9-beb5-1950dc835fb2 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:16.176298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b0577ab6-6797-4bb9-beb5-1950dc835fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:16.246805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527977e-9bcd-4508-9ee2-8020b46eb56c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:16.247685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 141/276] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 09 13:01:16.287595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c3518-9851-4d27-9911-b3e84945bd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:16.288136 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c3518-9851-4d27-9911-b3e84945bd0a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.288415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c3518-9851-4d27-9911-b3e84945bd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.288691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c3518-9851-4d27-9911-b3e84945bd0a None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:16.289365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c3518-9851-4d27-9911-b3e84945bd0a None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:16.289730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c3518-9851-4d27-9911-b3e84945bd0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:16.291241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c3518-9851-4d27-9911-b3e84945bd0a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:16.300393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 136/277] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:16.305856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0577ab6-6797-4bb9-beb5-1950dc835fb2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:16.415258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0577ab6-6797-4bb9-beb5-1950dc835fb2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:16.429417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:16.434988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c3518-9851-4d27-9911-b3e84945bd0a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:16.461506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a01d3-b753-4c73-b2e0-a070642868ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:16.463846 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a01d3-b753-4c73-b2e0-a070642868ff None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.464422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a01d3-b753-4c73-b2e0-a070642868ff None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.465153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a01d3-b753-4c73-b2e0-a070642868ff None admin] PATH_INFO: `/v3/users/3dae5044fd0f47b6b3f1581464903ca0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:16.484181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a01d3-b753-4c73-b2e0-a070642868ff None admin] RBAC: Authorizing `identity:get_user(user_id=3dae5044fd0f47b6b3f1581464903ca0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:16.486744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a01d3-b753-4c73-b2e0-a070642868ff None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:16.489215 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 142/278] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:16 2022] GET /identity/v3/users/3dae5044fd0f47b6b3f1581464903ca0 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:16.495756 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a01d3-b753-4c73-b2e0-a070642868ff None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:16.506022 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:16.554764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c3518-9851-4d27-9911-b3e84945bd0a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:16.596210 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36922ab7-09d4-41ec-a5b3-db61efcf5b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:16.596879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36922ab7-09d4-41ec-a5b3-db61efcf5b63 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.597379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36922ab7-09d4-41ec-a5b3-db61efcf5b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.597655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36922ab7-09d4-41ec-a5b3-db61efcf5b63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:16.599488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36922ab7-09d4-41ec-a5b3-db61efcf5b63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:16.601913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36922ab7-09d4-41ec-a5b3-db61efcf5b63 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:16.607962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 137/279] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:16.608251 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a01d3-b753-4c73-b2e0-a070642868ff None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:16.608371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36922ab7-09d4-41ec-a5b3-db61efcf5b63 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:16.640069 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7e5c9-002d-4db9-ae78-48432b4b6c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:16.640922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7e5c9-002d-4db9-ae78-48432b4b6c37 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.640922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7e5c9-002d-4db9-ae78-48432b4b6c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.641209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7e5c9-002d-4db9-ae78-48432b4b6c37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:16.641931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7e5c9-002d-4db9-ae78-48432b4b6c37 None admin] RBAC: Adding query filter params (role.id=e7b95eb3b5ad4479b50aaa68e58591ea, user.id=3dae5044fd0f47b6b3f1581464903ca0, scope.system=all) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:16.642237 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7e5c9-002d-4db9-ae78-48432b4b6c37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:16.643754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7e5c9-002d-4db9-ae78-48432b4b6c37 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:16.648943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 143/280] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Sep 9 13:01:16 2022] GET /identity/v3/role_assignments?user.id=3dae5044fd0f47b6b3f1581464903ca0&scope.system=all&role.id=e7b95eb3b5ad4479b50aaa68e58591ea => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:16.649534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7e5c9-002d-4db9-ae78-48432b4b6c37 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:16.695464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:16.732147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36922ab7-09d4-41ec-a5b3-db61efcf5b63 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:16.771128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:16.778643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5292e8-b052-4498-9370-85e7aefccd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:16.779756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5292e8-b052-4498-9370-85e7aefccd75 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.780625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5292e8-b052-4498-9370-85e7aefccd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.781018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5292e8-b052-4498-9370-85e7aefccd75 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:16.783378 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7e5c9-002d-4db9-ae78-48432b4b6c37 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:16.800241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5292e8-b052-4498-9370-85e7aefccd75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ee5292e8-b052-4498-9370-85e7aefccd75 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:16.808661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:16.809585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:16.810377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:16.810377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:16.810377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:16.810377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:16.810377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:16.810377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:16.810377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:16.810377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:16.810377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:16.810377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:16.810377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:16.810377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:16.811109 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 138/281] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:16.818337 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5292e8-b052-4498-9370-85e7aefccd75 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:16.831484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da116ec1-bbb7-4b62-ab73-bcf4eee29bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:16.832235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da116ec1-bbb7-4b62-ab73-bcf4eee29bee None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.832524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da116ec1-bbb7-4b62-ab73-bcf4eee29bee None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.832801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da116ec1-bbb7-4b62-ab73-bcf4eee29bee None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:16.833520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da116ec1-bbb7-4b62-ab73-bcf4eee29bee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:16.835086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da116ec1-bbb7-4b62-ab73-bcf4eee29bee None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:16.867864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 144/282] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:01:16 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:01:16.869239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a903b-165d-4e07-9f69-1e336d342489 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.869635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a903b-165d-4e07-9f69-1e336d342489 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.870522 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a903b-165d-4e07-9f69-1e336d342489 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:16.872429 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 145/283] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:16.878493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408fb66-219c-4eed-bdf0-04b0f9554c84 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.878791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408fb66-219c-4eed-bdf0-04b0f9554c84 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.879136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408fb66-219c-4eed-bdf0-04b0f9554c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:16.899353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0408fb66-219c-4eed-bdf0-04b0f9554c84 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:16.900848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0408fb66-219c-4eed-bdf0-04b0f9554c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:16.955594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5292e8-b052-4498-9370-85e7aefccd75 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:16.992616 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 146/284] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:01:16.998415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1caa61b-e14e-4404-b13a-8256e7f75cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:16.999032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1caa61b-e14e-4404-b13a-8256e7f75cec None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:16.999335 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1caa61b-e14e-4404-b13a-8256e7f75cec None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:16.999603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1caa61b-e14e-4404-b13a-8256e7f75cec None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:17.000328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1caa61b-e14e-4404-b13a-8256e7f75cec None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:17.000637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1caa61b-e14e-4404-b13a-8256e7f75cec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:17.002024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1caa61b-e14e-4404-b13a-8256e7f75cec None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:17.012160 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 139/285] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:17.017869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408fb66-219c-4eed-bdf0-04b0f9554c84 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:17.043917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:17.132643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408fb66-219c-4eed-bdf0-04b0f9554c84 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:17.169959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8805975-944f-467d-9888-463219e7da7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:17.170854 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8805975-944f-467d-9888-463219e7da7e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:17.171293 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8805975-944f-467d-9888-463219e7da7e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:17.171721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8805975-944f-467d-9888-463219e7da7e None admin] PATH_INFO: `/v3/users/27d14ad794d24c01b7a424f22564a745` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:17.173431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8805975-944f-467d-9888-463219e7da7e None admin] RBAC: Authorizing `identity:get_user(user_id=27d14ad794d24c01b7a424f22564a745)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:17.175115 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8805975-944f-467d-9888-463219e7da7e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:17.176707 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 147/286] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:17 2022] GET /identity/v3/users/27d14ad794d24c01b7a424f22564a745 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:17.183905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1caa61b-e14e-4404-b13a-8256e7f75cec None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:17.252629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:17.269506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8805975-944f-467d-9888-463219e7da7e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:17.308989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1caa61b-e14e-4404-b13a-8256e7f75cec None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:17.352564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea8e59-076c-4b0e-885c-88b34a0835b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:17.353843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea8e59-076c-4b0e-885c-88b34a0835b7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:17.354131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea8e59-076c-4b0e-885c-88b34a0835b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:17.354406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea8e59-076c-4b0e-885c-88b34a0835b7 None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:17.355887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea8e59-076c-4b0e-885c-88b34a0835b7 None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:17.357642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea8e59-076c-4b0e-885c-88b34a0835b7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:17.360111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 140/287] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:17 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:17.367756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea8e59-076c-4b0e-885c-88b34a0835b7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:17.396558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:17.412206 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8805975-944f-467d-9888-463219e7da7e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:17.452511 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd0710-a9d0-43f3-807e-eff5f982e343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:17.453156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd0710-a9d0-43f3-807e-eff5f982e343 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:17.453436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd0710-a9d0-43f3-807e-eff5f982e343 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:17.453707 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd0710-a9d0-43f3-807e-eff5f982e343 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:17.467532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd0710-a9d0-43f3-807e-eff5f982e343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0fcd0710-a9d0-43f3-807e-eff5f982e343 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:17.472514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:17.473434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:17.474224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:17.474224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:17.474224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:17.474224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:17.474224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:17.474224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:17.474224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:17.474224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:17.474224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:17.474224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:17.474224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:17.474224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:17.474224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 148/288] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:01:17.481296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd0710-a9d0-43f3-807e-eff5f982e343 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:17.503874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea8e59-076c-4b0e-885c-88b34a0835b7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:17.530358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:17.543877 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b5157-31b0-4e59-8a3f-7ea848a85644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:17.544549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b5157-31b0-4e59-8a3f-7ea848a85644 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:17.544867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b5157-31b0-4e59-8a3f-7ea848a85644 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:17.545146 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b5157-31b0-4e59-8a3f-7ea848a85644 None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a/users/27d14ad794d24c01b7a424f22564a745/roles/fb6fc2065aaa436db8a8d1d8c7f60ade` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:17.547326 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b5157-31b0-4e59-8a3f-7ea848a85644 None admin] RBAC: Authorizing `identity:create_grant(project_id=02dd689032dc41219d425e2f5ccadc7a, user_id=27d14ad794d24c01b7a424f22564a745, role_id=fb6fc2065aaa436db8a8d1d8c7f60ade)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:17.548813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b5157-31b0-4e59-8a3f-7ea848a85644 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:17.571272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 141/289] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:01:17 2022] PUT /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a/users/27d14ad794d24c01b7a424f22564a745/roles/fb6fc2065aaa436db8a8d1d8c7f60ade => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:01:17.605868 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd0710-a9d0-43f3-807e-eff5f982e343 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:17.646869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecc06a-fc62-44b4-8b6c-48dd53def396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:17.647878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecc06a-fc62-44b4-8b6c-48dd53def396 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:17.648361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecc06a-fc62-44b4-8b6c-48dd53def396 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:17.648902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecc06a-fc62-44b4-8b6c-48dd53def396 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:17.649789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecc06a-fc62-44b4-8b6c-48dd53def396 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:17.650436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecc06a-fc62-44b4-8b6c-48dd53def396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:17.652115 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecc06a-fc62-44b4-8b6c-48dd53def396 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:17.662588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 149/290] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:17.669584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b5157-31b0-4e59-8a3f-7ea848a85644 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:17.798552 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b5157-31b0-4e59-8a3f-7ea848a85644 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:17.847683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c060c5b-fd3a-410b-92f4-08a9f2952287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:17.848704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c060c5b-fd3a-410b-92f4-08a9f2952287 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:17.849186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c060c5b-fd3a-410b-92f4-08a9f2952287 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:17.849681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c060c5b-fd3a-410b-92f4-08a9f2952287 None admin] PATH_INFO: `/v3/users/57c677087d104d879d42eae0473a5e79` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:17.851031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c060c5b-fd3a-410b-92f4-08a9f2952287 None admin] RBAC: Authorizing `identity:get_user(user_id=57c677087d104d879d42eae0473a5e79)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:17.852824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c060c5b-fd3a-410b-92f4-08a9f2952287 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:17.855394 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 142/291] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:17 2022] GET /identity/v3/users/57c677087d104d879d42eae0473a5e79 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:17.861282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecc06a-fc62-44b4-8b6c-48dd53def396 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:17.875428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:17.972213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecc06a-fc62-44b4-8b6c-48dd53def396 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:18.008398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4284c1-4b5a-4f3b-82f1-a0e5a58287d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:18.009022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4284c1-4b5a-4f3b-82f1-a0e5a58287d6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:18.009306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4284c1-4b5a-4f3b-82f1-a0e5a58287d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:18.009776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4284c1-4b5a-4f3b-82f1-a0e5a58287d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:18.081362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4284c1-4b5a-4f3b-82f1-a0e5a58287d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:18.082828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4284c1-4b5a-4f3b-82f1-a0e5a58287d6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:18.095136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:18.189826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 150/292] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:18.195931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c060c5b-fd3a-410b-92f4-08a9f2952287 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:18.250495 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:18.322455 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c060c5b-fd3a-410b-92f4-08a9f2952287 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:18.365730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effd411-85ba-47b6-bb1f-d618893f0172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:18.366372 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effd411-85ba-47b6-bb1f-d618893f0172 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:18.366657 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effd411-85ba-47b6-bb1f-d618893f0172 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:18.366932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effd411-85ba-47b6-bb1f-d618893f0172 None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:18.368167 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effd411-85ba-47b6-bb1f-d618893f0172 None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:18.369650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effd411-85ba-47b6-bb1f-d618893f0172 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:18.371322 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 143/293] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:18 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:18.380360 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4284c1-4b5a-4f3b-82f1-a0e5a58287d6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:18.485655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4284c1-4b5a-4f3b-82f1-a0e5a58287d6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:18.498678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:18.524849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47914f2-bbf0-401b-aebe-d34161a0df98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:18.525449 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47914f2-bbf0-401b-aebe-d34161a0df98 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:18.525729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47914f2-bbf0-401b-aebe-d34161a0df98 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:18.526004 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47914f2-bbf0-401b-aebe-d34161a0df98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:18.526739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47914f2-bbf0-401b-aebe-d34161a0df98 None admin] RBAC: Adding query filter params (role.id=e7b95eb3b5ad4479b50aaa68e58591ea, scope.project.id=02dd689032dc41219d425e2f5ccadc7a, user.id=57c677087d104d879d42eae0473a5e79) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:18.527061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47914f2-bbf0-401b-aebe-d34161a0df98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:18.528521 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47914f2-bbf0-401b-aebe-d34161a0df98 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:18.534863 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 151/294] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:18 2022] GET /identity/v3/role_assignments?user.id=57c677087d104d879d42eae0473a5e79&scope.project.id=02dd689032dc41219d425e2f5ccadc7a&role.id=e7b95eb3b5ad4479b50aaa68e58591ea => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:19.265751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c3a09-2757-4880-a607-042fc80f138e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:19.266266 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c3a09-2757-4880-a607-042fc80f138e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:19.266372 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c3a09-2757-4880-a607-042fc80f138e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:19.267949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 144/295] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:19.273324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca7245-d02d-451f-b151-572e7ae0bf39 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:19.273746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca7245-d02d-451f-b151-572e7ae0bf39 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:19.273915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca7245-d02d-451f-b151-572e7ae0bf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:19.295396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-edca7245-d02d-451f-b151-572e7ae0bf39 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:19.296599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-edca7245-d02d-451f-b151-572e7ae0bf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:19.373274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 152/296] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:19.615846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98fbe5-4cac-4244-947a-ad17f0158b5b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:19.616387 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98fbe5-4cac-4244-947a-ad17f0158b5b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:19.616864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98fbe5-4cac-4244-947a-ad17f0158b5b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:19.618432 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 145/297] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:19.624515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdd662-63c3-42b6-bba0-f19c86ecd9b3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:19.625051 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdd662-63c3-42b6-bba0-f19c86ecd9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:19.625523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdd662-63c3-42b6-bba0-f19c86ecd9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:19.645017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-90fdd662-63c3-42b6-bba0-f19c86ecd9b3 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:19.646518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-90fdd662-63c3-42b6-bba0-f19c86ecd9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:19.648157 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98fbe5-4cac-4244-947a-ad17f0158b5b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:19.735097 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 153/298] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:19.784964 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98fbe5-4cac-4244-947a-ad17f0158b5b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:19.806824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:19.826523 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f663b42-8aa2-4628-9b3a-940f10acf960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:19.827454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f663b42-8aa2-4628-9b3a-940f10acf960 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:19.827933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f663b42-8aa2-4628-9b3a-940f10acf960 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:19.828350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f663b42-8aa2-4628-9b3a-940f10acf960 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:19.840377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f663b42-8aa2-4628-9b3a-940f10acf960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0f663b42-8aa2-4628-9b3a-940f10acf960 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:19.845623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:19.846737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:19.847823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:19.847823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:19.847823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:19.847823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:19.847823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:19.847823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:19.847823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:19.847823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:19.847823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:19.847823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:19.847823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:19.847823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:19.850102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 146/299] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:01:19.856108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdd662-63c3-42b6-bba0-f19c86ecd9b3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:19.893134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:19.905496 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de12a5-4503-4172-900a-20c152ebae13 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:19.906308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de12a5-4503-4172-900a-20c152ebae13 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:19.906766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de12a5-4503-4172-900a-20c152ebae13 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:19.909199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 147/300] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:19.914514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a9369-6c31-4780-b421-9f56de545f99 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:19.914798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a9369-6c31-4780-b421-9f56de545f99 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:19.915093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a9369-6c31-4780-b421-9f56de545f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:19.940637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-927a9369-6c31-4780-b421-9f56de545f99 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:19.941862 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-927a9369-6c31-4780-b421-9f56de545f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:19.973260 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdd662-63c3-42b6-bba0-f19c86ecd9b3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:20.017181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0c73a-11ca-4c80-82fa-7ef35f6b6cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:20.017794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0c73a-11ca-4c80-82fa-7ef35f6b6cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:20.018070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0c73a-11ca-4c80-82fa-7ef35f6b6cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:20.018336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0c73a-11ca-4c80-82fa-7ef35f6b6cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:20.018994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0c73a-11ca-4c80-82fa-7ef35f6b6cb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:20.019319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0c73a-11ca-4c80-82fa-7ef35f6b6cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:20.020693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0c73a-11ca-4c80-82fa-7ef35f6b6cb0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:20.036957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 154/301] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:20.036957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0c73a-11ca-4c80-82fa-7ef35f6b6cb0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:20.043513 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 148/302] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:01:20.047518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a9369-6c31-4780-b421-9f56de545f99 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:20.144520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:20.151544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0c73a-11ca-4c80-82fa-7ef35f6b6cb0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:20.159654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a9369-6c31-4780-b421-9f56de545f99 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:20.171276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:20.192885 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71a5fb-0131-4f92-ae16-fdbf80f49431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:20.193761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71a5fb-0131-4f92-ae16-fdbf80f49431 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:20.194043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71a5fb-0131-4f92-ae16-fdbf80f49431 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:20.194316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71a5fb-0131-4f92-ae16-fdbf80f49431 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:20.204931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f54209-3279-438f-b377-e320a10bc38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:20.205925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f54209-3279-438f-b377-e320a10bc38b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:20.206397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f54209-3279-438f-b377-e320a10bc38b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:20.206881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f54209-3279-438f-b377-e320a10bc38b None admin] PATH_INFO: `/v3/users/3dae5044fd0f47b6b3f1581464903ca0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:20.207811 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71a5fb-0131-4f92-ae16-fdbf80f49431 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:20.208528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f54209-3279-438f-b377-e320a10bc38b None admin] RBAC: Authorizing `identity:get_user(user_id=3dae5044fd0f47b6b3f1581464903ca0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:20.210211 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f54209-3279-438f-b377-e320a10bc38b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0f71a5fb-0131-4f92-ae16-fdbf80f49431 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:20.215101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:20.216024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:20.216834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:20.216834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:20.216834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:20.216834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:20.216834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:20.216834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:20.216834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:20.216834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:20.216834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:20.216834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:20.216834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:20.216834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:20.217466 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 149/303] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:20 2022] GET /identity/v3/users/3dae5044fd0f47b6b3f1581464903ca0 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:20.218324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f54209-3279-438f-b377-e320a10bc38b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:20.219454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 155/304] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:20.220566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71a5fb-0131-4f92-ae16-fdbf80f49431 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:20.328309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f54209-3279-438f-b377-e320a10bc38b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:20.332679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71a5fb-0131-4f92-ae16-fdbf80f49431 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:20.363815 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663a67b-5fb6-4c78-b289-0d441f8b15cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:20.364462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663a67b-5fb6-4c78-b289-0d441f8b15cb None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:20.364768 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663a67b-5fb6-4c78-b289-0d441f8b15cb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:20.365020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663a67b-5fb6-4c78-b289-0d441f8b15cb None admin] PATH_INFO: `/v3/system/users/3dae5044fd0f47b6b3f1581464903ca0/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:20.366399 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663a67b-5fb6-4c78-b289-0d441f8b15cb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3dae5044fd0f47b6b3f1581464903ca0, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:20.366945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bba4d6-3a2e-4600-b236-f799495d3bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:20.367579 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bba4d6-3a2e-4600-b236-f799495d3bee None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:20.367825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663a67b-5fb6-4c78-b289-0d441f8b15cb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:20.367935 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bba4d6-3a2e-4600-b236-f799495d3bee None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:20.368112 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bba4d6-3a2e-4600-b236-f799495d3bee None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:20.368839 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bba4d6-3a2e-4600-b236-f799495d3bee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:20.369124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bba4d6-3a2e-4600-b236-f799495d3bee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:20.370605 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bba4d6-3a2e-4600-b236-f799495d3bee None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:20.376200 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 150/305] 10.222.0.17 () {58 vars in 1334 bytes} [Fri Sep 9 13:01:20 2022] PUT /identity/v3/system/users/3dae5044fd0f47b6b3f1581464903ca0/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:01:20.387585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ed8ce-3bc8-4fc0-9586-2801e0532346 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:20.387585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ed8ce-3bc8-4fc0-9586-2801e0532346 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:20.387585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ed8ce-3bc8-4fc0-9586-2801e0532346 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:20.387585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 151/306] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:20.387918 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 156/307] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:20.388012 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ed8ce-3bc8-4fc0-9586-2801e0532346 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:20.388096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17110e-0187-476b-b1e3-16b155d819cb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:20.388096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17110e-0187-476b-b1e3-16b155d819cb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:20.388096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17110e-0187-476b-b1e3-16b155d819cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:20.408501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1e17110e-0187-476b-b1e3-16b155d819cb None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:20.409772 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1e17110e-0187-476b-b1e3-16b155d819cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:20.425508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:20.436032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:20.492260 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 157/308] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:20.500210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17110e-0187-476b-b1e3-16b155d819cb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:20.515005 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ed8ce-3bc8-4fc0-9586-2801e0532346 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:20.557559 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853bc65-e723-420e-bf88-0a1d8fdafb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:20.558347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853bc65-e723-420e-bf88-0a1d8fdafb33 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:20.558626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853bc65-e723-420e-bf88-0a1d8fdafb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:20.558897 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853bc65-e723-420e-bf88-0a1d8fdafb33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:20.560415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853bc65-e723-420e-bf88-0a1d8fdafb33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:20.562455 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853bc65-e723-420e-bf88-0a1d8fdafb33 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:20.565620 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 152/309] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:20.570883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853bc65-e723-420e-bf88-0a1d8fdafb33 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:20.640777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17110e-0187-476b-b1e3-16b155d819cb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:20.676406 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e3e35-435c-40a3-97ff-06151529226e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:20.677050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e3e35-435c-40a3-97ff-06151529226e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:20.677327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e3e35-435c-40a3-97ff-06151529226e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:20.677600 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e3e35-435c-40a3-97ff-06151529226e None admin] PATH_INFO: `/v3/users/456129e8c0074bf1aeeb5f9c975f338c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:20.691565 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e3e35-435c-40a3-97ff-06151529226e None admin] RBAC: Authorizing `identity:get_user(user_id=456129e8c0074bf1aeeb5f9c975f338c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:20.693005 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e3e35-435c-40a3-97ff-06151529226e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:20.695273 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 158/310] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:20 2022] GET /identity/v3/users/456129e8c0074bf1aeeb5f9c975f338c => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:20.704468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e3e35-435c-40a3-97ff-06151529226e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:20.716013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853bc65-e723-420e-bf88-0a1d8fdafb33 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:20.727890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:20.729347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:20.763844 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bb677-d99d-4cee-8198-715085349611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:20.764780 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bb677-d99d-4cee-8198-715085349611 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:20.765066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bb677-d99d-4cee-8198-715085349611 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:20.765341 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bb677-d99d-4cee-8198-715085349611 None admin] PATH_INFO: `/v3/groups` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:20.765981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095bb677-d99d-4cee-8198-715085349611 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:20.767945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095bb677-d99d-4cee-8198-715085349611 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:20.784076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 153/311] 10.222.0.17 () {60 vars in 1134 bytes} [Fri Sep 9 13:01:20 2022] POST /identity/v3/groups => generated 229 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:01:20.792156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bb677-d99d-4cee-8198-715085349611 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:20.822561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e3e35-435c-40a3-97ff-06151529226e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:20.867983 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b765dca-0f24-4b04-999f-027b553b086d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:20.869190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b765dca-0f24-4b04-999f-027b553b086d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:20.869650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b765dca-0f24-4b04-999f-027b553b086d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:20.869933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b765dca-0f24-4b04-999f-027b553b086d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:20.870704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b765dca-0f24-4b04-999f-027b553b086d None admin] RBAC: Adding query filter params (role.id=fb6fc2065aaa436db8a8d1d8c7f60ade, user.id=456129e8c0074bf1aeeb5f9c975f338c, scope.system=all) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:20.871015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b765dca-0f24-4b04-999f-027b553b086d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:20.872527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b765dca-0f24-4b04-999f-027b553b086d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:20.881753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 159/312] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Sep 9 13:01:20 2022] GET /identity/v3/role_assignments?user.id=456129e8c0074bf1aeeb5f9c975f338c&scope.system=all&role.id=fb6fc2065aaa436db8a8d1d8c7f60ade => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:20.935595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bb677-d99d-4cee-8198-715085349611 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:20.979530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7db17-1990-4522-836a-af29f8395341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:20.980196 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7db17-1990-4522-836a-af29f8395341 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:20.980476 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7db17-1990-4522-836a-af29f8395341 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:20.980750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7db17-1990-4522-836a-af29f8395341 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:20.994284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7db17-1990-4522-836a-af29f8395341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7db7db17-1990-4522-836a-af29f8395341 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:21.000024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:21.000950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:21.001739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:21.001739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:21.001739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:21.001739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:21.001739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:21.001739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:21.001739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:21.001739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:21.001739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:21.001739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:21.001739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:21.001739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:21.001739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 154/313] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:21.008544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b765dca-0f24-4b04-999f-027b553b086d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:21.092331 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:21.125661 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b765dca-0f24-4b04-999f-027b553b086d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:21.162642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0be4a-0085-4ea8-9a70-468fa966e7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:21.163723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0be4a-0085-4ea8-9a70-468fa966e7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:21.164202 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0be4a-0085-4ea8-9a70-468fa966e7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:21.164666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0be4a-0085-4ea8-9a70-468fa966e7e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:21.165798 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0be4a-0085-4ea8-9a70-468fa966e7e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:21.166301 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0be4a-0085-4ea8-9a70-468fa966e7e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:21.167925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0be4a-0085-4ea8-9a70-468fa966e7e7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:21.177016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 160/314] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:21.183660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7db17-1990-4522-836a-af29f8395341 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:21.218033 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:21.292651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7db17-1990-4522-836a-af29f8395341 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:21.329200 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fb0e5-866c-4afb-9c10-ccfa6fb81c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:21.330728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fb0e5-866c-4afb-9c10-ccfa6fb81c2b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:21.331247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fb0e5-866c-4afb-9c10-ccfa6fb81c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:21.331734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fb0e5-866c-4afb-9c10-ccfa6fb81c2b None admin] PATH_INFO: `/v3/users/27d14ad794d24c01b7a424f22564a745` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:21.333286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fb0e5-866c-4afb-9c10-ccfa6fb81c2b None admin] RBAC: Authorizing `identity:get_user(user_id=27d14ad794d24c01b7a424f22564a745)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:21.335127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fb0e5-866c-4afb-9c10-ccfa6fb81c2b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:21.337954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 155/315] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:21 2022] GET /identity/v3/users/27d14ad794d24c01b7a424f22564a745 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:21.344127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0be4a-0085-4ea8-9a70-468fa966e7e7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:21.450563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0be4a-0085-4ea8-9a70-468fa966e7e7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:21.481060 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:21.487718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496e8da-de81-4f5a-ac9b-96998bf54802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:21.488314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496e8da-de81-4f5a-ac9b-96998bf54802 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:21.488956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496e8da-de81-4f5a-ac9b-96998bf54802 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:21.489262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496e8da-de81-4f5a-ac9b-96998bf54802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:21.561214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496e8da-de81-4f5a-ac9b-96998bf54802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:21.562702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496e8da-de81-4f5a-ac9b-96998bf54802 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:21.668815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 161/316] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:21.674101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fb0e5-866c-4afb-9c10-ccfa6fb81c2b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:21.772614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:21.779148 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fb0e5-866c-4afb-9c10-ccfa6fb81c2b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:21.817064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263fd59-6561-4f49-b94c-ed10fa5ca6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:21.817726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263fd59-6561-4f49-b94c-ed10fa5ca6bc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:21.818007 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263fd59-6561-4f49-b94c-ed10fa5ca6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:21.818608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263fd59-6561-4f49-b94c-ed10fa5ca6bc None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:21.819872 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263fd59-6561-4f49-b94c-ed10fa5ca6bc None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:21.821965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263fd59-6561-4f49-b94c-ed10fa5ca6bc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:21.824081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 156/317] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:21 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:21.830192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496e8da-de81-4f5a-ac9b-96998bf54802 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:21.860621 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:21.929070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496e8da-de81-4f5a-ac9b-96998bf54802 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:21.964322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892ca46-e05d-4785-a8bd-3b648d2b8b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:21.964930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892ca46-e05d-4785-a8bd-3b648d2b8b7d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:21.965207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892ca46-e05d-4785-a8bd-3b648d2b8b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:21.965478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892ca46-e05d-4785-a8bd-3b648d2b8b7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:21.966201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892ca46-e05d-4785-a8bd-3b648d2b8b7d None admin] RBAC: Adding query filter params (role.id=fb6fc2065aaa436db8a8d1d8c7f60ade, scope.project.id=02dd689032dc41219d425e2f5ccadc7a, user.id=27d14ad794d24c01b7a424f22564a745) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:21.966508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892ca46-e05d-4785-a8bd-3b648d2b8b7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:21.967977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892ca46-e05d-4785-a8bd-3b648d2b8b7d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:21.973721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 162/318] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:21 2022] GET /identity/v3/role_assignments?user.id=27d14ad794d24c01b7a424f22564a745&scope.project.id=02dd689032dc41219d425e2f5ccadc7a&role.id=fb6fc2065aaa436db8a8d1d8c7f60ade => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:22.648942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bad681-d51f-49a2-840a-ee5bb4b7c579 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:22.649945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bad681-d51f-49a2-840a-ee5bb4b7c579 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:22.650156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bad681-d51f-49a2-840a-ee5bb4b7c579 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:22.651961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 157/319] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:22.657383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d4024-33dd-48d8-9d29-1fbc1affa3e3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:22.657790 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d4024-33dd-48d8-9d29-1fbc1affa3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:22.657926 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d4024-33dd-48d8-9d29-1fbc1affa3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:22.676625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2f3d4024-33dd-48d8-9d29-1fbc1affa3e3 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:22.677796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2f3d4024-33dd-48d8-9d29-1fbc1affa3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:22.747091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 163/320] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:23.022405 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bad681-d51f-49a2-840a-ee5bb4b7c579 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:23.135665 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bad681-d51f-49a2-840a-ee5bb4b7c579 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:23.150822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c65528-fc39-4158-92f4-68d4bd1c2d43 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:23.151147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c65528-fc39-4158-92f4-68d4bd1c2d43 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:23.151425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c65528-fc39-4158-92f4-68d4bd1c2d43 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:23.155264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 164/321] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:01:23.157938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429292-2468-41e7-a863-ed3faaeba122 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:23.158237 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429292-2468-41e7-a863-ed3faaeba122 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:23.158515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429292-2468-41e7-a863-ed3faaeba122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:23.169600 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c236e96-4166-4f4a-a655-1a3548f93683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:23.170244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c236e96-4166-4f4a-a655-1a3548f93683 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:23.170525 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c236e96-4166-4f4a-a655-1a3548f93683 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:23.170805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c236e96-4166-4f4a-a655-1a3548f93683 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:23.176996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-bd429292-2468-41e7-a863-ed3faaeba122 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:23.178206 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bd429292-2468-41e7-a863-ed3faaeba122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:23.181252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c236e96-4166-4f4a-a655-1a3548f93683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9c236e96-4166-4f4a-a655-1a3548f93683 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:23.186543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:23.187675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 158/322] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:23.193574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77ad0e-506c-4581-a010-113a2938621f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:23.194277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77ad0e-506c-4581-a010-113a2938621f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:23.194277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77ad0e-506c-4581-a010-113a2938621f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:23.194277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 159/323] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:23.194277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77ad0e-506c-4581-a010-113a2938621f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:23.203011 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:23.220379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:23.250743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 165/324] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:01:23.252096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1346b-378c-4a92-afc6-d5bdf4e24393 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:23.252385 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1346b-378c-4a92-afc6-d5bdf4e24393 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:23.252659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1346b-378c-4a92-afc6-d5bdf4e24393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:23.270305 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a7f1346b-378c-4a92-afc6-d5bdf4e24393 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:23.271815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a7f1346b-378c-4a92-afc6-d5bdf4e24393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:23.304752 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77ad0e-506c-4581-a010-113a2938621f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:23.342571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942a503-f1e0-46ba-9c40-33e907cbe419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:23.343216 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942a503-f1e0-46ba-9c40-33e907cbe419 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:23.343386 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 166/325] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:23.343483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942a503-f1e0-46ba-9c40-33e907cbe419 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:23.343757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942a503-f1e0-46ba-9c40-33e907cbe419 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:23.344419 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942a503-f1e0-46ba-9c40-33e907cbe419 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:23.344724 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942a503-f1e0-46ba-9c40-33e907cbe419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:23.346110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942a503-f1e0-46ba-9c40-33e907cbe419 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:23.359829 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 160/326] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:23.360855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1346b-378c-4a92-afc6-d5bdf4e24393 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:23.443323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:23.458763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1346b-378c-4a92-afc6-d5bdf4e24393 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:23.492379 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343c822-c5af-4b70-9173-2781f25811ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:23.493145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343c822-c5af-4b70-9173-2781f25811ab None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:23.493422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343c822-c5af-4b70-9173-2781f25811ab None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:23.493779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343c822-c5af-4b70-9173-2781f25811ab None admin] PATH_INFO: `/v3/users/3dae5044fd0f47b6b3f1581464903ca0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:23.494908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343c822-c5af-4b70-9173-2781f25811ab None admin] RBAC: Authorizing `identity:get_user(user_id=3dae5044fd0f47b6b3f1581464903ca0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:23.496375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343c822-c5af-4b70-9173-2781f25811ab None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:23.498110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 167/327] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:23 2022] GET /identity/v3/users/3dae5044fd0f47b6b3f1581464903ca0 => generated 293 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:23.503728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942a503-f1e0-46ba-9c40-33e907cbe419 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:23.525832 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343c822-c5af-4b70-9173-2781f25811ab None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:23.596008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:23.615560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942a503-f1e0-46ba-9c40-33e907cbe419 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:23.641126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343c822-c5af-4b70-9173-2781f25811ab None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:23.649596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde46277-90c8-4e72-ba0b-de63369999ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:23.650211 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde46277-90c8-4e72-ba0b-de63369999ef None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:23.650487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde46277-90c8-4e72-ba0b-de63369999ef None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:23.650759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde46277-90c8-4e72-ba0b-de63369999ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:23.651516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde46277-90c8-4e72-ba0b-de63369999ef None admin] RBAC: Adding query filter params (role.id=e7b95eb3b5ad4479b50aaa68e58591ea, user.id=3dae5044fd0f47b6b3f1581464903ca0, scope.system=all) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:23.651831 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde46277-90c8-4e72-ba0b-de63369999ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:23.653309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde46277-90c8-4e72-ba0b-de63369999ef None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:23.664575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 161/328] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Sep 9 13:01:23 2022] GET /identity/v3/role_assignments?user.id=3dae5044fd0f47b6b3f1581464903ca0&scope.system=all&role.id=e7b95eb3b5ad4479b50aaa68e58591ea => generated 510 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:23.665890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde46277-90c8-4e72-ba0b-de63369999ef None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:23.687777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc141641-0bdc-41b6-8568-5ee3a604dcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:23.688457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc141641-0bdc-41b6-8568-5ee3a604dcfa None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:23.688707 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc141641-0bdc-41b6-8568-5ee3a604dcfa None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:23.688980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc141641-0bdc-41b6-8568-5ee3a604dcfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:23.690181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc141641-0bdc-41b6-8568-5ee3a604dcfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:23.691715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc141641-0bdc-41b6-8568-5ee3a604dcfa None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:23.693853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 168/329] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:23.700196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc141641-0bdc-41b6-8568-5ee3a604dcfa None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:23.727041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:23.780318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde46277-90c8-4e72-ba0b-de63369999ef None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:23.812800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc141641-0bdc-41b6-8568-5ee3a604dcfa None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:23.816345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85774c-1d8a-4702-bd17-e3d75fcd859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:23.817003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85774c-1d8a-4702-bd17-e3d75fcd859d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:23.817283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85774c-1d8a-4702-bd17-e3d75fcd859d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:23.817558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85774c-1d8a-4702-bd17-e3d75fcd859d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:23.830261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85774c-1d8a-4702-bd17-e3d75fcd859d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-cc85774c-1d8a-4702-bd17-e3d75fcd859d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:23.835594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:23.836490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:23.838168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:23.838168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:23.838168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:23.838168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:23.838168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:23.838168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:23.838168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:23.838168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:23.838168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:23.838168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:23.838168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:23.838168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:23.838168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 162/330] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:23.842912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85774c-1d8a-4702-bd17-e3d75fcd859d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:23.847038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:23.858058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f3026-c96d-4ecc-9f7c-3b6ca253e97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:23.858659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f3026-c96d-4ecc-9f7c-3b6ca253e97f None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:23.858943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f3026-c96d-4ecc-9f7c-3b6ca253e97f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:23.859285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f3026-c96d-4ecc-9f7c-3b6ca253e97f None admin] PATH_INFO: `/v3/groups` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:23.859895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f3026-c96d-4ecc-9f7c-3b6ca253e97f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:23.861339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f3026-c96d-4ecc-9f7c-3b6ca253e97f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:23.879957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 169/331] 10.222.0.17 () {60 vars in 1134 bytes} [Fri Sep 9 13:01:23 2022] POST /identity/v3/groups => generated 226 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:01:23.965821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85774c-1d8a-4702-bd17-e3d75fcd859d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:24.001776 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4578ad-6b9c-4945-9998-fc5dd051123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:24.002394 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4578ad-6b9c-4945-9998-fc5dd051123a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:24.002668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4578ad-6b9c-4945-9998-fc5dd051123a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:24.002938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4578ad-6b9c-4945-9998-fc5dd051123a None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:24.003646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4578ad-6b9c-4945-9998-fc5dd051123a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:24.003949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4578ad-6b9c-4945-9998-fc5dd051123a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:24.005358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4578ad-6b9c-4945-9998-fc5dd051123a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:24.015148 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 163/332] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:24.020907 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f3026-c96d-4ecc-9f7c-3b6ca253e97f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:24.107346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:24.129048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f3026-c96d-4ecc-9f7c-3b6ca253e97f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:24.165454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f072b-fb84-4774-821f-ee83f2ee48fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:24.166103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f072b-fb84-4774-821f-ee83f2ee48fe None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:24.166380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f072b-fb84-4774-821f-ee83f2ee48fe None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:24.166653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f072b-fb84-4774-821f-ee83f2ee48fe None admin] PATH_INFO: `/v3/users/456129e8c0074bf1aeeb5f9c975f338c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:24.167847 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f072b-fb84-4774-821f-ee83f2ee48fe None admin] RBAC: Authorizing `identity:get_user(user_id=456129e8c0074bf1aeeb5f9c975f338c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:24.169305 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f072b-fb84-4774-821f-ee83f2ee48fe None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:24.171092 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 170/333] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:24 2022] GET /identity/v3/users/456129e8c0074bf1aeeb5f9c975f338c => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:24.177460 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4578ad-6b9c-4945-9998-fc5dd051123a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:24.251254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:24.294643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4578ad-6b9c-4945-9998-fc5dd051123a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:24.335666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091407ca-a8b4-4ef2-9d2c-4d7edd48d597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:24.336333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091407ca-a8b4-4ef2-9d2c-4d7edd48d597 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:24.336648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091407ca-a8b4-4ef2-9d2c-4d7edd48d597 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:24.336930 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091407ca-a8b4-4ef2-9d2c-4d7edd48d597 None admin] PATH_INFO: `/v3/system/users/456129e8c0074bf1aeeb5f9c975f338c/roles/fb6fc2065aaa436db8a8d1d8c7f60ade` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:24.338559 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091407ca-a8b4-4ef2-9d2c-4d7edd48d597 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=456129e8c0074bf1aeeb5f9c975f338c, role_id=fb6fc2065aaa436db8a8d1d8c7f60ade)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:24.339997 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091407ca-a8b4-4ef2-9d2c-4d7edd48d597 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:24.349024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 164/334] 10.222.0.17 () {58 vars in 1334 bytes} [Fri Sep 9 13:01:24 2022] PUT /identity/v3/system/users/456129e8c0074bf1aeeb5f9c975f338c/roles/fb6fc2065aaa436db8a8d1d8c7f60ade => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:01:26.708865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a50490-7bd2-455c-862c-919a1b9950ea None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:26.709358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a50490-7bd2-455c-862c-919a1b9950ea None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:26.709432 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a50490-7bd2-455c-862c-919a1b9950ea None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:26.711143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 171/335] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:26.716801 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01fcad-9d0b-4617-aa22-4652f04b0409 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:26.717235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01fcad-9d0b-4617-aa22-4652f04b0409 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:26.717423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01fcad-9d0b-4617-aa22-4652f04b0409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:26.725287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6be36c-db57-470c-b48c-cf9adfb9c620 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:26.725819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6be36c-db57-470c-b48c-cf9adfb9c620 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:26.726323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6be36c-db57-470c-b48c-cf9adfb9c620 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:26.728636 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 172/336] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:26.734030 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad5b3-3ee6-458a-93c8-016f383cd234 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:26.734563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad5b3-3ee6-458a-93c8-016f383cd234 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:26.735073 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad5b3-3ee6-458a-93c8-016f383cd234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:26.747848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3c01fcad-9d0b-4617-aa22-4652f04b0409 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:26.756400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3c01fcad-9d0b-4617-aa22-4652f04b0409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:26.777215 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c87ad5b3-3ee6-458a-93c8-016f383cd234 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:26.778757 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c87ad5b3-3ee6-458a-93c8-016f383cd234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:26.901834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 165/337] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:01:26.936279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 173/338] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 09 13:01:27.120933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a34501-722b-446d-810b-7f0b3bb605e9 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:27.121607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a34501-722b-446d-810b-7f0b3bb605e9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:27.122086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a34501-722b-446d-810b-7f0b3bb605e9 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:27.123947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 166/339] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:27.129076 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7be8-3a5d-4075-8a18-0ba4db36958b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:27.129360 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7be8-3a5d-4075-8a18-0ba4db36958b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:27.129804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7be8-3a5d-4075-8a18-0ba4db36958b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:27.160139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-adef7be8-3a5d-4075-8a18-0ba4db36958b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:27.161580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-adef7be8-3a5d-4075-8a18-0ba4db36958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:27.252279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 174/340] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:27.316646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8048adf-0b25-4488-8084-903130229165 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:27.317066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8048adf-0b25-4488-8084-903130229165 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:27.317700 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8048adf-0b25-4488-8084-903130229165 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:27.319471 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 167/341] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:27.326694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbdc34-6995-4ca8-9977-f161c0570792 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:27.327208 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbdc34-6995-4ca8-9977-f161c0570792 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:27.327628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbdc34-6995-4ca8-9977-f161c0570792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:27.352588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-77dbdc34-6995-4ca8-9977-f161c0570792 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:27.354053 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-77dbdc34-6995-4ca8-9977-f161c0570792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:27.357925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:27.387107 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8048adf-0b25-4488-8084-903130229165 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:27.479272 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 175/342] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:27.504969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:27.556689 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8048adf-0b25-4488-8084-903130229165 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:27.599074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36343482-2479-405e-8eb0-77db24b2185f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:27.599074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36343482-2479-405e-8eb0-77db24b2185f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:27.599074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36343482-2479-405e-8eb0-77db24b2185f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:27.599074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36343482-2479-405e-8eb0-77db24b2185f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:27.605524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbdc34-6995-4ca8-9977-f161c0570792 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:27.611995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36343482-2479-405e-8eb0-77db24b2185f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-36343482-2479-405e-8eb0-77db24b2185f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:27.623099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:27.624038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:27.624866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:27.624866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:27.624866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:27.624866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:27.624866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:27.624866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:27.624866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:27.624866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:27.624866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:27.624866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:27.624866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:27.624866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:27.624866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 168/343] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:01:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:01:27.627344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36343482-2479-405e-8eb0-77db24b2185f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:27.753164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36343482-2479-405e-8eb0-77db24b2185f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:27.777516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:27.794280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2354d8a-c62a-492b-8de3-830308fb07c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:27.794894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2354d8a-c62a-492b-8de3-830308fb07c6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:27.795488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2354d8a-c62a-492b-8de3-830308fb07c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:27.798864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2354d8a-c62a-492b-8de3-830308fb07c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:27.798864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2354d8a-c62a-492b-8de3-830308fb07c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:27.798864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2354d8a-c62a-492b-8de3-830308fb07c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:27.798864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2354d8a-c62a-492b-8de3-830308fb07c6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:27.807583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 169/344] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:27.808418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2354d8a-c62a-492b-8de3-830308fb07c6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:27.828386 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbdc34-6995-4ca8-9977-f161c0570792 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:27.902733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9e8fc-cb16-4228-a1c5-3ecbe66c63b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:27.903402 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9e8fc-cb16-4228-a1c5-3ecbe66c63b1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:27.903680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9e8fc-cb16-4228-a1c5-3ecbe66c63b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:27.903951 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9e8fc-cb16-4228-a1c5-3ecbe66c63b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:27.912763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:27.918375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9e8fc-cb16-4228-a1c5-3ecbe66c63b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-00b9e8fc-cb16-4228-a1c5-3ecbe66c63b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:27.923518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:27.924391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:27.925193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:27.925193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:27.925193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:27.925193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:27.925193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:27.925193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:27.925193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:27.925193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:27.925193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:27.925193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:27.925193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:27.925193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:27.927755 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 176/345] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:27.928865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9e8fc-cb16-4228-a1c5-3ecbe66c63b1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:27.931413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2354d8a-c62a-492b-8de3-830308fb07c6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:27.972945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827597c7-741a-4b9f-8290-e55efacae531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:27.973926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827597c7-741a-4b9f-8290-e55efacae531 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:27.974497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827597c7-741a-4b9f-8290-e55efacae531 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:27.975076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827597c7-741a-4b9f-8290-e55efacae531 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:27.989578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827597c7-741a-4b9f-8290-e55efacae531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-827597c7-741a-4b9f-8290-e55efacae531 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:27.995850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:27.996791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:27.997605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:27.997605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:27.997605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:27.997605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:27.997605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:27.997605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:27.997605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:27.997605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:27.997605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:27.997605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:27.997605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:27.997605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:28.000380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 170/346] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:28.001517 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827597c7-741a-4b9f-8290-e55efacae531 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:28.056633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:28.067756 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9e8fc-cb16-4228-a1c5-3ecbe66c63b1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:28.110789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cbe2f-9896-436f-a0b1-05fab86a7ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:28.111447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cbe2f-9896-436f-a0b1-05fab86a7ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:28.111729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cbe2f-9896-436f-a0b1-05fab86a7ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:28.112007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cbe2f-9896-436f-a0b1-05fab86a7ee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:28.128508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827597c7-741a-4b9f-8290-e55efacae531 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:28.171897 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0d852-f8c7-4097-a35e-53513d2956eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:28.172983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0d852-f8c7-4097-a35e-53513d2956eb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:28.173457 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0d852-f8c7-4097-a35e-53513d2956eb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:28.174004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0d852-f8c7-4097-a35e-53513d2956eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:28.186612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0d852-f8c7-4097-a35e-53513d2956eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:28.189407 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:28.191925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cbe2f-9896-436f-a0b1-05fab86a7ee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:28.193653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cbe2f-9896-436f-a0b1-05fab86a7ee0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e3f0d852-f8c7-4097-a35e-53513d2956eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:28.202763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:28.203746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:28.204545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:28.204545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:28.204545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:28.204545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:28.204545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:28.204545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:28.204545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:28.204545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:28.204545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:28.204545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:28.204545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 13:01:28.204545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:28.207173 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 171/347] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:28.208168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0d852-f8c7-4097-a35e-53513d2956eb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:28.327470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 177/348] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:28.328303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cbe2f-9896-436f-a0b1-05fab86a7ee0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:28.339568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0d852-f8c7-4097-a35e-53513d2956eb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:28.340958 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:28.390517 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e209e-b86b-4be8-919b-8580696decdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:28.391339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e209e-b86b-4be8-919b-8580696decdd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:28.391620 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e209e-b86b-4be8-919b-8580696decdd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:28.391893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e209e-b86b-4be8-919b-8580696decdd None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:28.392625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e209e-b86b-4be8-919b-8580696decdd None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:28.392941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e209e-b86b-4be8-919b-8580696decdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:28.394342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e209e-b86b-4be8-919b-8580696decdd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:28.407237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 172/349] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:28.408430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e209e-b86b-4be8-919b-8580696decdd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:28.461082 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:28.464362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cbe2f-9896-436f-a0b1-05fab86a7ee0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:28.511509 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f12a7-7de2-4516-878d-0fe82ebd5aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:28.512134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f12a7-7de2-4516-878d-0fe82ebd5aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:28.512409 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f12a7-7de2-4516-878d-0fe82ebd5aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:28.512679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f12a7-7de2-4516-878d-0fe82ebd5aa9 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:28.513351 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f12a7-7de2-4516-878d-0fe82ebd5aa9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:28.513651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f12a7-7de2-4516-878d-0fe82ebd5aa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:28.515023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f12a7-7de2-4516-878d-0fe82ebd5aa9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:28.526409 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 178/350] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:28.527527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e42bfe-c88a-44b5-a148-28948471708f None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:28.527806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e42bfe-c88a-44b5-a148-28948471708f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:28.528076 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e42bfe-c88a-44b5-a148-28948471708f None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:28.529395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 179/351] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:28.530747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e42bfe-c88a-44b5-a148-28948471708f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:28.542319 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e209e-b86b-4be8-919b-8580696decdd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:28.588459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644e0ef-d3d2-4800-bc11-9619f2e6ff44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:28.588995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644e0ef-d3d2-4800-bc11-9619f2e6ff44 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:28.589314 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644e0ef-d3d2-4800-bc11-9619f2e6ff44 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:28.589584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644e0ef-d3d2-4800-bc11-9619f2e6ff44 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:28.590248 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644e0ef-d3d2-4800-bc11-9619f2e6ff44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:28.593255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644e0ef-d3d2-4800-bc11-9619f2e6ff44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:28.593255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644e0ef-d3d2-4800-bc11-9619f2e6ff44 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:28.602203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 173/352] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:28.609762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644e0ef-d3d2-4800-bc11-9619f2e6ff44 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:28.654717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:28.674018 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e42bfe-c88a-44b5-a148-28948471708f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:28.723264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210ea94-44bf-4bee-b27f-1fc418f09b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:28.724285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210ea94-44bf-4bee-b27f-1fc418f09b4e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:28.724765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210ea94-44bf-4bee-b27f-1fc418f09b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:28.725226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210ea94-44bf-4bee-b27f-1fc418f09b4e None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:28.727641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210ea94-44bf-4bee-b27f-1fc418f09b4e None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:28.729276 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210ea94-44bf-4bee-b27f-1fc418f09b4e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:28.731274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 180/353] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:28 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:28.732250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210ea94-44bf-4bee-b27f-1fc418f09b4e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:28.764602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644e0ef-d3d2-4800-bc11-9619f2e6ff44 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:28.772601 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:28.807194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fdb4c-57db-4d71-a357-fc78a5c6ec6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:28.807822 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fdb4c-57db-4d71-a357-fc78a5c6ec6a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:28.808101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fdb4c-57db-4d71-a357-fc78a5c6ec6a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:28.808483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fdb4c-57db-4d71-a357-fc78a5c6ec6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:28.863700 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210ea94-44bf-4bee-b27f-1fc418f09b4e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:28.893153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fdb4c-57db-4d71-a357-fc78a5c6ec6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:28.894914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fdb4c-57db-4d71-a357-fc78a5c6ec6a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:28.901808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456a9cd-35f5-4d3c-b3b8-76b907781363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:28.902428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456a9cd-35f5-4d3c-b3b8-76b907781363 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:28.902701 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456a9cd-35f5-4d3c-b3b8-76b907781363 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:28.902968 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456a9cd-35f5-4d3c-b3b8-76b907781363 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:28.925153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:28.984731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456a9cd-35f5-4d3c-b3b8-76b907781363 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:28.986196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456a9cd-35f5-4d3c-b3b8-76b907781363 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:29.017369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 174/354] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:29.018449 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fa119-3031-4043-97c9-6ed97bfe9d5f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:29.018735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fa119-3031-4043-97c9-6ed97bfe9d5f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:29.019010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fa119-3031-4043-97c9-6ed97bfe9d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:29.022927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:29.039371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-be0fa119-3031-4043-97c9-6ed97bfe9d5f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:29.040761 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-be0fa119-3031-4043-97c9-6ed97bfe9d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:29.114834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 181/355] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:29.116094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456a9cd-35f5-4d3c-b3b8-76b907781363 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:29.127538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 175/356] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:29.128370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fa119-3031-4043-97c9-6ed97bfe9d5f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:29.199172 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:29.246839 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456a9cd-35f5-4d3c-b3b8-76b907781363 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:29.253964 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fa119-3031-4043-97c9-6ed97bfe9d5f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:29.284730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:29.293248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2694bb-fb2d-4017-802e-a0a565c09b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:29.293882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2694bb-fb2d-4017-802e-a0a565c09b88 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:29.294197 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2694bb-fb2d-4017-802e-a0a565c09b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:29.294489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2694bb-fb2d-4017-802e-a0a565c09b88 None admin] PATH_INFO: `/v3/users/456129e8c0074bf1aeeb5f9c975f338c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:29.295658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2694bb-fb2d-4017-802e-a0a565c09b88 None admin] RBAC: Authorizing `identity:get_user(user_id=456129e8c0074bf1aeeb5f9c975f338c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:29.297038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2694bb-fb2d-4017-802e-a0a565c09b88 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:29.298812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e02b81-56cc-401c-a383-e2628330ee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:29.298949 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 182/357] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:01:29 2022] GET /identity/v3/users/456129e8c0074bf1aeeb5f9c975f338c => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:29.299264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e02b81-56cc-401c-a383-e2628330ee21 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:29.299546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e02b81-56cc-401c-a383-e2628330ee21 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:29.299815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef018da-976d-4cc6-a4f7-ba7149f05723 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:29.299916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e02b81-56cc-401c-a383-e2628330ee21 None admin] PATH_INFO: `/v3/groups/ae53619a7e634d7caf14b7d3511f43ed` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:29.300091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef018da-976d-4cc6-a4f7-ba7149f05723 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:29.300359 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef018da-976d-4cc6-a4f7-ba7149f05723 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:29.301725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 183/358] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:29.302401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef018da-976d-4cc6-a4f7-ba7149f05723 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:29.310893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e02b81-56cc-401c-a383-e2628330ee21 None admin] RBAC: Authorizing `identity:get_group(group_id=ae53619a7e634d7caf14b7d3511f43ed)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:29.312396 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e02b81-56cc-401c-a383-e2628330ee21 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:29.314347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 176/359] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:29 2022] GET /identity/v3/groups/ae53619a7e634d7caf14b7d3511f43ed => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:29.315153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e02b81-56cc-401c-a383-e2628330ee21 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:29.436981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e02b81-56cc-401c-a383-e2628330ee21 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:29.444132 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef018da-976d-4cc6-a4f7-ba7149f05723 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:29.461919 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:29.480587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b1493-2cba-498b-b578-956f66219391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:29.481221 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b1493-2cba-498b-b578-956f66219391 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:29.481487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b1493-2cba-498b-b578-956f66219391 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:29.482568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b1493-2cba-498b-b578-956f66219391 None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:29.484036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b1493-2cba-498b-b578-956f66219391 None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:29.485493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b1493-2cba-498b-b578-956f66219391 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:29.485608 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb37aa-1da6-4cde-8d2e-09c6659a4f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:29.486256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb37aa-1da6-4cde-8d2e-09c6659a4f6f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:29.486541 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb37aa-1da6-4cde-8d2e-09c6659a4f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:29.486812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb37aa-1da6-4cde-8d2e-09c6659a4f6f None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:29.487147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 177/360] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:29 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:29.488292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6fe34-506b-4791-ba99-95702cecbc83 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:29.488540 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6fe34-506b-4791-ba99-95702cecbc83 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:29.488807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6fe34-506b-4791-ba99-95702cecbc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:29.492358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb37aa-1da6-4cde-8d2e-09c6659a4f6f None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:29.492358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb37aa-1da6-4cde-8d2e-09c6659a4f6f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:29.493755 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 184/361] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:29 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:29.498187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb37aa-1da6-4cde-8d2e-09c6659a4f6f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:29.514724 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8ef6fe34-506b-4791-ba99-95702cecbc83 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:29.515970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8ef6fe34-506b-4791-ba99-95702cecbc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:29.571965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:29.599886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 178/362] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:29.600751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6fe34-506b-4791-ba99-95702cecbc83 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:29.622985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb37aa-1da6-4cde-8d2e-09c6659a4f6f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:29.676020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68251d-4682-4167-b37e-1e4382fdb3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:29.676634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68251d-4682-4167-b37e-1e4382fdb3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:29.677216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68251d-4682-4167-b37e-1e4382fdb3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:29.677216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68251d-4682-4167-b37e-1e4382fdb3c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:29.677882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68251d-4682-4167-b37e-1e4382fdb3c3 None admin] RBAC: Adding query filter params (role.id=fb6fc2065aaa436db8a8d1d8c7f60ade, user.id=456129e8c0074bf1aeeb5f9c975f338c, scope.system=all) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:29.678191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68251d-4682-4167-b37e-1e4382fdb3c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:29.679913 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68251d-4682-4167-b37e-1e4382fdb3c3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:29.699376 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 185/363] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Sep 9 13:01:29 2022] GET /identity/v3/role_assignments?user.id=456129e8c0074bf1aeeb5f9c975f338c&scope.system=all&role.id=fb6fc2065aaa436db8a8d1d8c7f60ade => generated 510 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:29.700904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68251d-4682-4167-b37e-1e4382fdb3c3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:29.735073 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6fe34-506b-4791-ba99-95702cecbc83 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:29.764822 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:29.782503 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e11327-ce52-41de-9822-60df0c3c7adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:29.783163 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e11327-ce52-41de-9822-60df0c3c7adf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:29.783778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e11327-ce52-41de-9822-60df0c3c7adf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:29.784058 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e11327-ce52-41de-9822-60df0c3c7adf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:29.785106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e11327-ce52-41de-9822-60df0c3c7adf None admin] RBAC: Adding query filter params (group.id=ae53619a7e634d7caf14b7d3511f43ed, role.id=0238ab71fb17471cb87e84097272c752, scope.project.id=482e7368e31f415faa7b8b65d04ae50f) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:29.785424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e11327-ce52-41de-9822-60df0c3c7adf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:29.787475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e11327-ce52-41de-9822-60df0c3c7adf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:29.796309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 179/364] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 13:01:29 2022] GET /identity/v3/role_assignments?group.id=ae53619a7e634d7caf14b7d3511f43ed&scope.project.id=482e7368e31f415faa7b8b65d04ae50f&role.id=0238ab71fb17471cb87e84097272c752 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:29.797252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e11327-ce52-41de-9822-60df0c3c7adf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:29.876391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68251d-4682-4167-b37e-1e4382fdb3c3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:29.917629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:29.930784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5c741-67b6-4b14-8ce2-4166f13e5750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:29.931416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5c741-67b6-4b14-8ce2-4166f13e5750 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:29.931705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5c741-67b6-4b14-8ce2-4166f13e5750 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:29.931985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5c741-67b6-4b14-8ce2-4166f13e5750 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:29.940454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e11327-ce52-41de-9822-60df0c3c7adf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:29.950341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5c741-67b6-4b14-8ce2-4166f13e5750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-46e5c741-67b6-4b14-8ce2-4166f13e5750 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:29.956908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:29.957848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 186/365] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:01:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:29.958673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5c741-67b6-4b14-8ce2-4166f13e5750 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:30.000128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c74279-34d6-4f61-9895-f248475ea16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:30.001362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c74279-34d6-4f61-9895-f248475ea16a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:30.001718 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c74279-34d6-4f61-9895-f248475ea16a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:30.002708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c74279-34d6-4f61-9895-f248475ea16a None admin] PATH_INFO: `/v3/groups/ae53619a7e634d7caf14b7d3511f43ed` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:30.003629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c74279-34d6-4f61-9895-f248475ea16a None admin] RBAC: Authorizing `identity:get_group(group_id=ae53619a7e634d7caf14b7d3511f43ed)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:30.006015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c74279-34d6-4f61-9895-f248475ea16a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:30.009391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 180/366] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:29 2022] GET /identity/v3/groups/ae53619a7e634d7caf14b7d3511f43ed => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:30.009391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c74279-34d6-4f61-9895-f248475ea16a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:30.087318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:30.096382 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5c741-67b6-4b14-8ce2-4166f13e5750 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:30.146199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff44d38-7660-4bfc-9b72-5acb80b860c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:30.147341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff44d38-7660-4bfc-9b72-5acb80b860c4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:30.148653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff44d38-7660-4bfc-9b72-5acb80b860c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:30.149127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff44d38-7660-4bfc-9b72-5acb80b860c4 None admin] PATH_INFO: `/v3/groups/ae53619a7e634d7caf14b7d3511f43ed` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:30.151035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff44d38-7660-4bfc-9b72-5acb80b860c4 None admin] RBAC: Authorizing `identity:get_group(group_id=ae53619a7e634d7caf14b7d3511f43ed)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:30.152535 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff44d38-7660-4bfc-9b72-5acb80b860c4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:30.158859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 187/367] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:29 2022] GET /identity/v3/groups/ae53619a7e634d7caf14b7d3511f43ed => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:30.159012 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c74279-34d6-4f61-9895-f248475ea16a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:30.159122 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff44d38-7660-4bfc-9b72-5acb80b860c4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:30.201800 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dffa41-ef4a-43f1-9cd9-06fe4490ba46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:30.202368 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dffa41-ef4a-43f1-9cd9-06fe4490ba46 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:30.202649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dffa41-ef4a-43f1-9cd9-06fe4490ba46 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:30.202928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dffa41-ef4a-43f1-9cd9-06fe4490ba46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:30.220227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dffa41-ef4a-43f1-9cd9-06fe4490ba46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:30.223562 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-77dffa41-ef4a-43f1-9cd9-06fe4490ba46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:30.225797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:30.227526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:30.228345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:30.228345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:30.228345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:30.228345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:30.228345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:30.228345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:30.228345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:30.228345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:30.228345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:30.228345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:30.228345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:30.228345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:30.228345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 181/368] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:01:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:30.228982 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dffa41-ef4a-43f1-9cd9-06fe4490ba46 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:30.303504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff44d38-7660-4bfc-9b72-5acb80b860c4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:30.346236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8d2ff-68e7-402e-971c-fa4498c4b4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:30.347090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8d2ff-68e7-402e-971c-fa4498c4b4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:30.347230 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8d2ff-68e7-402e-971c-fa4498c4b4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:30.347432 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8d2ff-68e7-402e-971c-fa4498c4b4f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:30.348106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8d2ff-68e7-402e-971c-fa4498c4b4f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:30.348411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8d2ff-68e7-402e-971c-fa4498c4b4f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:30.351467 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8d2ff-68e7-402e-971c-fa4498c4b4f2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:30.361966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 188/369] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:30.362147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8d2ff-68e7-402e-971c-fa4498c4b4f2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:30.372448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dffa41-ef4a-43f1-9cd9-06fe4490ba46 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:30.386192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:30.421965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cb9e4-3182-4cd0-bb7b-117bcbdd4256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:30.422633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cb9e4-3182-4cd0-bb7b-117bcbdd4256 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:30.422878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cb9e4-3182-4cd0-bb7b-117bcbdd4256 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:30.424035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cb9e4-3182-4cd0-bb7b-117bcbdd4256 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:30.424322 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cb9e4-3182-4cd0-bb7b-117bcbdd4256 None admin] RBAC: Adding query filter params (group.id=ae53619a7e634d7caf14b7d3511f43ed, role.id=e7b95eb3b5ad4479b50aaa68e58591ea, scope.project.id=02dd689032dc41219d425e2f5ccadc7a) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:30.424632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cb9e4-3182-4cd0-bb7b-117bcbdd4256 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:30.427818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cb9e4-3182-4cd0-bb7b-117bcbdd4256 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:30.433185 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 182/370] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 13:01:30 2022] GET /identity/v3/role_assignments?group.id=ae53619a7e634d7caf14b7d3511f43ed&scope.project.id=02dd689032dc41219d425e2f5ccadc7a&role.id=e7b95eb3b5ad4479b50aaa68e58591ea => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:30.435200 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cb9e4-3182-4cd0-bb7b-117bcbdd4256 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:30.521210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8d2ff-68e7-402e-971c-fa4498c4b4f2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:30.549517 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:30.582186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59601785-e2fe-474d-b6b5-2530a055269b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:30.583541 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59601785-e2fe-474d-b6b5-2530a055269b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:30.584042 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59601785-e2fe-474d-b6b5-2530a055269b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:30.584520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59601785-e2fe-474d-b6b5-2530a055269b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:30.585542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59601785-e2fe-474d-b6b5-2530a055269b None admin] RBAC: Adding query filter params (group.id=ae53619a7e634d7caf14b7d3511f43ed, role.id=e7b95eb3b5ad4479b50aaa68e58591ea, scope.project.id=482e7368e31f415faa7b8b65d04ae50f) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:30.586071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59601785-e2fe-474d-b6b5-2530a055269b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:30.587812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59601785-e2fe-474d-b6b5-2530a055269b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:30.588453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cb9e4-3182-4cd0-bb7b-117bcbdd4256 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:30.599502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 189/371] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 13:01:30 2022] GET /identity/v3/role_assignments?group.id=ae53619a7e634d7caf14b7d3511f43ed&scope.project.id=482e7368e31f415faa7b8b65d04ae50f&role.id=e7b95eb3b5ad4479b50aaa68e58591ea => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:30.610850 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59601785-e2fe-474d-b6b5-2530a055269b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:30.670853 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f5c36-830a-4021-b8d2-7e1138af38f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:30.671510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f5c36-830a-4021-b8d2-7e1138af38f9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:30.671835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f5c36-830a-4021-b8d2-7e1138af38f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:30.672104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f5c36-830a-4021-b8d2-7e1138af38f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:30.672783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f5c36-830a-4021-b8d2-7e1138af38f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:30.673084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f5c36-830a-4021-b8d2-7e1138af38f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:30.678732 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f5c36-830a-4021-b8d2-7e1138af38f9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:30.695283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 183/372] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:30.699893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f5c36-830a-4021-b8d2-7e1138af38f9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:30.750469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:30.833257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f5c36-830a-4021-b8d2-7e1138af38f9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:30.833851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59601785-e2fe-474d-b6b5-2530a055269b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:30.878424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61b24f-4cb7-48a3-9a08-f0450f12d22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:30.879174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61b24f-4cb7-48a3-9a08-f0450f12d22d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:30.879555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61b24f-4cb7-48a3-9a08-f0450f12d22d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:30.879924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61b24f-4cb7-48a3-9a08-f0450f12d22d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:30.929943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d160d-5f8a-40d1-bf76-595205210b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:30.930832 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d160d-5f8a-40d1-bf76-595205210b15 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:30.931137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d160d-5f8a-40d1-bf76-595205210b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:30.931415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d160d-5f8a-40d1-bf76-595205210b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:30.979907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61b24f-4cb7-48a3-9a08-f0450f12d22d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:30.982553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61b24f-4cb7-48a3-9a08-f0450f12d22d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:31.014766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:31.047003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:31.058521 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d160d-5f8a-40d1-bf76-595205210b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:31.060027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d160d-5f8a-40d1-bf76-595205210b15 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:31.171311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 184/373] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 13:01:31.173472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61b24f-4cb7-48a3-9a08-f0450f12d22d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:31.232685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 190/374] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:31.240316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d160d-5f8a-40d1-bf76-595205210b15 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:31.350790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61b24f-4cb7-48a3-9a08-f0450f12d22d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:31.359483 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:31.384098 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d160d-5f8a-40d1-bf76-595205210b15 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:31.387363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76918413-25b1-4a5b-9eca-a250d9d2f3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:31.388009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76918413-25b1-4a5b-9eca-a250d9d2f3cf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:31.388293 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76918413-25b1-4a5b-9eca-a250d9d2f3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:31.388567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76918413-25b1-4a5b-9eca-a250d9d2f3cf None admin] PATH_INFO: `/v3/projects/97408104e346493ebafc4d549d2257c4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:31.389699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76918413-25b1-4a5b-9eca-a250d9d2f3cf None admin] RBAC: Authorizing `identity:get_project(project_id=97408104e346493ebafc4d549d2257c4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:31.391168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76918413-25b1-4a5b-9eca-a250d9d2f3cf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:31.393203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 185/375] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:31 2022] GET /identity/v3/projects/97408104e346493ebafc4d549d2257c4 => generated 344 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:31.398494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76918413-25b1-4a5b-9eca-a250d9d2f3cf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:31.418560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:31.421918 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6b219-a473-4273-9603-daeb648a658b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:31.422652 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6b219-a473-4273-9603-daeb648a658b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:31.422928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6b219-a473-4273-9603-daeb648a658b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:31.423245 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6b219-a473-4273-9603-daeb648a658b None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:31.424375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6b219-a473-4273-9603-daeb648a658b None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:31.429864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6b219-a473-4273-9603-daeb648a658b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:31.429864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 191/376] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:31 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:31.436220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6b219-a473-4273-9603-daeb648a658b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:31.541894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6b219-a473-4273-9603-daeb648a658b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:31.581232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb569a6-939d-4758-8a7d-951fc2cb51b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:31.582028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb569a6-939d-4758-8a7d-951fc2cb51b8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:31.582403 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb569a6-939d-4758-8a7d-951fc2cb51b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:31.582774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb569a6-939d-4758-8a7d-951fc2cb51b8 None admin] PATH_INFO: `/v3/groups/ae53619a7e634d7caf14b7d3511f43ed` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:31.587078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb569a6-939d-4758-8a7d-951fc2cb51b8 None admin] RBAC: Authorizing `identity:get_group(group_id=ae53619a7e634d7caf14b7d3511f43ed)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:31.587078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb569a6-939d-4758-8a7d-951fc2cb51b8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:31.587745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 192/377] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:31 2022] GET /identity/v3/groups/ae53619a7e634d7caf14b7d3511f43ed => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:31.594015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb569a6-939d-4758-8a7d-951fc2cb51b8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:31.626879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:31.658180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76918413-25b1-4a5b-9eca-a250d9d2f3cf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:31.719736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b3f8a-2a62-4c27-87f9-15f770d89e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:31.720302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b3f8a-2a62-4c27-87f9-15f770d89e61 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:31.720581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b3f8a-2a62-4c27-87f9-15f770d89e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:31.720851 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b3f8a-2a62-4c27-87f9-15f770d89e61 None admin] PATH_INFO: `/v3/groups/1c90f2746e2d447a862039de9dd6a014` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:31.735742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b3f8a-2a62-4c27-87f9-15f770d89e61 None admin] RBAC: Authorizing `identity:get_group(group_id=1c90f2746e2d447a862039de9dd6a014)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:31.742719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b3f8a-2a62-4c27-87f9-15f770d89e61 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:31.744772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 186/378] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:31 2022] GET /identity/v3/groups/1c90f2746e2d447a862039de9dd6a014 => generated 229 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:31.750547 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b3f8a-2a62-4c27-87f9-15f770d89e61 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:31.789049 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb569a6-939d-4758-8a7d-951fc2cb51b8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:31.833014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189a85a-a99b-49f0-9d2a-9004264e02c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:31.835066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189a85a-a99b-49f0-9d2a-9004264e02c7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:31.837260 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189a85a-a99b-49f0-9d2a-9004264e02c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:31.837696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189a85a-a99b-49f0-9d2a-9004264e02c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:31.838591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189a85a-a99b-49f0-9d2a-9004264e02c7 None admin] RBAC: Adding query filter params (group.id=ae53619a7e634d7caf14b7d3511f43ed, role.id=0238ab71fb17471cb87e84097272c752, scope.project.id=02dd689032dc41219d425e2f5ccadc7a) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:31.839368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189a85a-a99b-49f0-9d2a-9004264e02c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:31.841127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189a85a-a99b-49f0-9d2a-9004264e02c7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:31.854532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 193/379] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 13:01:31 2022] GET /identity/v3/role_assignments?group.id=ae53619a7e634d7caf14b7d3511f43ed&scope.project.id=02dd689032dc41219d425e2f5ccadc7a&role.id=0238ab71fb17471cb87e84097272c752 => generated 260 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:31.881660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:31.938677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b3f8a-2a62-4c27-87f9-15f770d89e61 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:31.998031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f510b7f-d06c-4ca1-8861-6a88f1f9063b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:31.998915 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f510b7f-d06c-4ca1-8861-6a88f1f9063b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:31.999379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f510b7f-d06c-4ca1-8861-6a88f1f9063b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:31.999795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f510b7f-d06c-4ca1-8861-6a88f1f9063b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:32.000684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f510b7f-d06c-4ca1-8861-6a88f1f9063b None admin] RBAC: Adding query filter params (group.id=1c90f2746e2d447a862039de9dd6a014, role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=97408104e346493ebafc4d549d2257c4) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:32.001164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f510b7f-d06c-4ca1-8861-6a88f1f9063b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:32.002857 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f510b7f-d06c-4ca1-8861-6a88f1f9063b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:32.012217 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 187/380] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 13:01:31 2022] GET /identity/v3/role_assignments?group.id=1c90f2746e2d447a862039de9dd6a014&scope.project.id=97408104e346493ebafc4d549d2257c4&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 260 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:32.670781 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314bd08-7070-4148-94d7-36dd837d648d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:32.671791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314bd08-7070-4148-94d7-36dd837d648d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:32.672264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314bd08-7070-4148-94d7-36dd837d648d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:32.674080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 194/381] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:32.680590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4f205-9572-413b-8987-b1776f96f9ca None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:32.681119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4f205-9572-413b-8987-b1776f96f9ca None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:32.681615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4f205-9572-413b-8987-b1776f96f9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:32.711898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9fe4f205-9572-413b-8987-b1776f96f9ca None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:32.713470 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9fe4f205-9572-413b-8987-b1776f96f9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:32.794400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 188/382] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:01:33.001344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0f91a-72ea-4b72-a1cd-2e1a1892d49c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:33.002044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0f91a-72ea-4b72-a1cd-2e1a1892d49c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:33.002527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0f91a-72ea-4b72-a1cd-2e1a1892d49c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:33.004359 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 195/383] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:33.009742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e549c0e-b0ef-4034-b9f7-8bac6aba5cd9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:33.010577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e549c0e-b0ef-4034-b9f7-8bac6aba5cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:33.011117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e549c0e-b0ef-4034-b9f7-8bac6aba5cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:33.033857 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4e549c0e-b0ef-4034-b9f7-8bac6aba5cd9 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:33.035480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4e549c0e-b0ef-4034-b9f7-8bac6aba5cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:33.119419 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:33.133206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 189/384] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:33.182888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde0bd3-557b-459d-a0aa-36126204c539 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:33.183552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde0bd3-557b-459d-a0aa-36126204c539 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:33.184045 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde0bd3-557b-459d-a0aa-36126204c539 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:33.185820 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 196/385] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:33.191384 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea6564-92a5-490b-96f3-61c39b823a8b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:33.192127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea6564-92a5-490b-96f3-61c39b823a8b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:33.192614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea6564-92a5-490b-96f3-61c39b823a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:33.217040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-92ea6564-92a5-490b-96f3-61c39b823a8b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:33.218534 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-92ea6564-92a5-490b-96f3-61c39b823a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:33.225402 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde0bd3-557b-459d-a0aa-36126204c539 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:33.347499 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 190/386] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:33.349113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:33.372494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde0bd3-557b-459d-a0aa-36126204c539 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:33.408324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893e7dc-c8c4-444f-ac4e-e0072c063399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:33.409188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893e7dc-c8c4-444f-ac4e-e0072c063399 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:33.409631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893e7dc-c8c4-444f-ac4e-e0072c063399 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:33.410066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893e7dc-c8c4-444f-ac4e-e0072c063399 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:33.421097 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893e7dc-c8c4-444f-ac4e-e0072c063399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7893e7dc-c8c4-444f-ac4e-e0072c063399 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:33.426501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:33.427597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:33.428424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:33.428424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:33.428424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:33.428424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:33.428424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:33.428424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:33.428424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:33.428424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:33.428424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:33.428424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:33.428424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:33.428424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:33.429138 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 197/387] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:01:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:33.434344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea6564-92a5-490b-96f3-61c39b823a8b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:33.445516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893e7dc-c8c4-444f-ac4e-e0072c063399 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:33.593076 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893e7dc-c8c4-444f-ac4e-e0072c063399 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:33.630491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e4e58-cbd5-4526-9c17-c97f5b7faa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:33.631500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e4e58-cbd5-4526-9c17-c97f5b7faa82 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:33.632742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e4e58-cbd5-4526-9c17-c97f5b7faa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:33.633248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e4e58-cbd5-4526-9c17-c97f5b7faa82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:33.640905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:33.645992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e4e58-cbd5-4526-9c17-c97f5b7faa82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:33.648972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea6564-92a5-490b-96f3-61c39b823a8b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-861e4e58-cbd5-4526-9c17-c97f5b7faa82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:33.653591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:33.654466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:33.655322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:33.655322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:33.655322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:33.655322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:33.655322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:33.655322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:33.655322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:33.655322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:33.655322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:33.655322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:33.655322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:33.655322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:33.657677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 198/388] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:01:33.663692 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e4e58-cbd5-4526-9c17-c97f5b7faa82 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:33.686048 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:33.700604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89875b2-3929-4850-a49c-d2ab0339ed0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:33.701217 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89875b2-3929-4850-a49c-d2ab0339ed0a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:33.701495 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89875b2-3929-4850-a49c-d2ab0339ed0a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:33.701764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89875b2-3929-4850-a49c-d2ab0339ed0a None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:33.702424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89875b2-3929-4850-a49c-d2ab0339ed0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:33.702725 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89875b2-3929-4850-a49c-d2ab0339ed0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:33.704429 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89875b2-3929-4850-a49c-d2ab0339ed0a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:33.716164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 191/389] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:33.717062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89875b2-3929-4850-a49c-d2ab0339ed0a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:33.780412 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e4e58-cbd5-4526-9c17-c97f5b7faa82 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:33.821195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120bad5-98b8-43d8-9f88-cbc2718ddb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:33.821902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120bad5-98b8-43d8-9f88-cbc2718ddb69 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:33.822252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120bad5-98b8-43d8-9f88-cbc2718ddb69 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:33.822601 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120bad5-98b8-43d8-9f88-cbc2718ddb69 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:33.823371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120bad5-98b8-43d8-9f88-cbc2718ddb69 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:33.823769 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120bad5-98b8-43d8-9f88-cbc2718ddb69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:33.825228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120bad5-98b8-43d8-9f88-cbc2718ddb69 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:33.839211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 199/390] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:33.840176 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120bad5-98b8-43d8-9f88-cbc2718ddb69 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:33.852475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89875b2-3929-4850-a49c-d2ab0339ed0a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:33.900418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504c7b0-5c5a-413d-bc2b-0f502ac0cc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:33.901906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504c7b0-5c5a-413d-bc2b-0f502ac0cc8e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:33.902367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504c7b0-5c5a-413d-bc2b-0f502ac0cc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:33.902975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504c7b0-5c5a-413d-bc2b-0f502ac0cc8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:33.908842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:33.919658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504c7b0-5c5a-413d-bc2b-0f502ac0cc8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e504c7b0-5c5a-413d-bc2b-0f502ac0cc8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:33.927119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:33.928036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:33.929473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:33.929473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:33.929473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:33.929473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:33.929473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:33.929473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:33.929473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:33.929473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:33.929473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:33.929473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:33.929473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:33.929473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:33.932304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 192/391] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:33.933558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504c7b0-5c5a-413d-bc2b-0f502ac0cc8e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:33.956828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120bad5-98b8-43d8-9f88-cbc2718ddb69 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:33.994749 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:34.014304 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a1e3b-7244-4040-bba9-561e1f3b4e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:34.014944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a1e3b-7244-4040-bba9-561e1f3b4e98 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.015250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a1e3b-7244-4040-bba9-561e1f3b4e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.015526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a1e3b-7244-4040-bba9-561e1f3b4e98 None admin] PATH_INFO: `/v3/groups/ae53619a7e634d7caf14b7d3511f43ed` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.017303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a1e3b-7244-4040-bba9-561e1f3b4e98 None admin] RBAC: Authorizing `identity:get_group(group_id=ae53619a7e634d7caf14b7d3511f43ed)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:34.018757 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a1e3b-7244-4040-bba9-561e1f3b4e98 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:34.020737 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 200/392] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:33 2022] GET /identity/v3/groups/ae53619a7e634d7caf14b7d3511f43ed => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:34.021655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a1e3b-7244-4040-bba9-561e1f3b4e98 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:34.065514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504c7b0-5c5a-413d-bc2b-0f502ac0cc8e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:34.104099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a88340-ffa0-432a-a3de-fd5a34383e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:34.105117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a88340-ffa0-432a-a3de-fd5a34383e94 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.105617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a88340-ffa0-432a-a3de-fd5a34383e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.106094 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a88340-ffa0-432a-a3de-fd5a34383e94 None admin] PATH_INFO: `/v3/groups/ae53619a7e634d7caf14b7d3511f43ed` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.108825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a88340-ffa0-432a-a3de-fd5a34383e94 None admin] RBAC: Authorizing `identity:get_group(group_id=ae53619a7e634d7caf14b7d3511f43ed)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:34.110645 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a88340-ffa0-432a-a3de-fd5a34383e94 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:34.112472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 193/393] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:33 2022] GET /identity/v3/groups/ae53619a7e634d7caf14b7d3511f43ed => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:34.113496 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a88340-ffa0-432a-a3de-fd5a34383e94 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:34.136469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a1e3b-7244-4040-bba9-561e1f3b4e98 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:34.164678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:34.174016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e4164-7f19-4e98-a56f-2076e6095e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:34.174634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e4164-7f19-4e98-a56f-2076e6095e28 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.174917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e4164-7f19-4e98-a56f-2076e6095e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.175266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e4164-7f19-4e98-a56f-2076e6095e28 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.175892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e4164-7f19-4e98-a56f-2076e6095e28 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:34.176200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e4164-7f19-4e98-a56f-2076e6095e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:34.177577 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e4164-7f19-4e98-a56f-2076e6095e28 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:34.186699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 201/394] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:34.187736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e4164-7f19-4e98-a56f-2076e6095e28 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:34.227818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a88340-ffa0-432a-a3de-fd5a34383e94 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:34.242144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:34.266466 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f9c21-96d8-4652-bdfd-8ee1b7bdbb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:34.267469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f9c21-96d8-4652-bdfd-8ee1b7bdbb71 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.267947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f9c21-96d8-4652-bdfd-8ee1b7bdbb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.268435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f9c21-96d8-4652-bdfd-8ee1b7bdbb71 None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.269763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f9c21-96d8-4652-bdfd-8ee1b7bdbb71 None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:34.271504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f9c21-96d8-4652-bdfd-8ee1b7bdbb71 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:34.273990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 194/395] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:34 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:34.275043 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f9c21-96d8-4652-bdfd-8ee1b7bdbb71 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:34.295996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e4164-7f19-4e98-a56f-2076e6095e28 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:34.332379 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c67d2-5ca6-44c1-9a89-9f274998797f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:34.333018 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c67d2-5ca6-44c1-9a89-9f274998797f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.333301 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c67d2-5ca6-44c1-9a89-9f274998797f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.333577 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c67d2-5ca6-44c1-9a89-9f274998797f None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.334754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c67d2-5ca6-44c1-9a89-9f274998797f None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:34.336418 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c67d2-5ca6-44c1-9a89-9f274998797f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:34.343006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 202/396] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:34 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:34.343844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c67d2-5ca6-44c1-9a89-9f274998797f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:34.386939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f9c21-96d8-4652-bdfd-8ee1b7bdbb71 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:34.406240 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:34.424833 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923fda8-af79-4fd3-8c5e-7cea289bd916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:34.425957 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923fda8-af79-4fd3-8c5e-7cea289bd916 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.426504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923fda8-af79-4fd3-8c5e-7cea289bd916 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.427009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923fda8-af79-4fd3-8c5e-7cea289bd916 None admin] PATH_INFO: `/v3/groups/ae53619a7e634d7caf14b7d3511f43ed` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.428596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923fda8-af79-4fd3-8c5e-7cea289bd916 None admin] RBAC: Authorizing `identity:get_group(group_id=ae53619a7e634d7caf14b7d3511f43ed)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:34.430923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923fda8-af79-4fd3-8c5e-7cea289bd916 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:34.432981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 195/397] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:34 2022] GET /identity/v3/groups/ae53619a7e634d7caf14b7d3511f43ed => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:34.434276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923fda8-af79-4fd3-8c5e-7cea289bd916 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:34.458982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c67d2-5ca6-44c1-9a89-9f274998797f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:34.494372 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:34.498370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74048f1f-4d13-4275-95ac-f812c196544d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:34.499179 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74048f1f-4d13-4275-95ac-f812c196544d None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.499535 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74048f1f-4d13-4275-95ac-f812c196544d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.499887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74048f1f-4d13-4275-95ac-f812c196544d None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f/groups/ae53619a7e634d7caf14b7d3511f43ed/roles/0238ab71fb17471cb87e84097272c752` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.502434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74048f1f-4d13-4275-95ac-f812c196544d None admin] RBAC: Authorizing `identity:create_grant(project_id=482e7368e31f415faa7b8b65d04ae50f, group_id=ae53619a7e634d7caf14b7d3511f43ed, role_id=0238ab71fb17471cb87e84097272c752)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:34.504068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74048f1f-4d13-4275-95ac-f812c196544d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:34.522888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 203/398] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 9 13:01:34 2022] PUT /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f/groups/ae53619a7e634d7caf14b7d3511f43ed/roles/0238ab71fb17471cb87e84097272c752 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:01:34.524119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74048f1f-4d13-4275-95ac-f812c196544d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:34.564940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923fda8-af79-4fd3-8c5e-7cea289bd916 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:34.605661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84a657-3b31-4999-ae23-54308a26346e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:34.606470 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84a657-3b31-4999-ae23-54308a26346e None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.606870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84a657-3b31-4999-ae23-54308a26346e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.607249 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84a657-3b31-4999-ae23-54308a26346e None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f/groups/ae53619a7e634d7caf14b7d3511f43ed/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.609926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84a657-3b31-4999-ae23-54308a26346e None admin] RBAC: Authorizing `identity:create_grant(project_id=482e7368e31f415faa7b8b65d04ae50f, group_id=ae53619a7e634d7caf14b7d3511f43ed, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:34.613812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84a657-3b31-4999-ae23-54308a26346e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:34.628931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 196/399] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 9 13:01:34 2022] PUT /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f/groups/ae53619a7e634d7caf14b7d3511f43ed/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:01:34.630369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c35c3-aeb7-425b-8256-c5b8d6eee023 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.630660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c35c3-aeb7-425b-8256-c5b8d6eee023 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.630939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c35c3-aeb7-425b-8256-c5b8d6eee023 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.633392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 197/400] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:34.634437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed51e03-6bb3-43c6-904b-96d8725f37d2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.634744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed51e03-6bb3-43c6-904b-96d8725f37d2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.634996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed51e03-6bb3-43c6-904b-96d8725f37d2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.636521 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 198/401] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:34.639940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619be01-03ca-4e93-9eb5-a8ff10ebc7bf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.640340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619be01-03ca-4e93-9eb5-a8ff10ebc7bf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.641651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619be01-03ca-4e93-9eb5-a8ff10ebc7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.663667 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4619be01-03ca-4e93-9eb5-a8ff10ebc7bf None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:34.665112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4619be01-03ca-4e93-9eb5-a8ff10ebc7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:34.671205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74048f1f-4d13-4275-95ac-f812c196544d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:34.697300 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:34.728689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d94920-4cc1-4ffe-9d27-847b8b22641c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:34.730318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d94920-4cc1-4ffe-9d27-847b8b22641c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.730618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d94920-4cc1-4ffe-9d27-847b8b22641c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.730898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d94920-4cc1-4ffe-9d27-847b8b22641c None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.732487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d94920-4cc1-4ffe-9d27-847b8b22641c None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:34.735109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d94920-4cc1-4ffe-9d27-847b8b22641c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:34.737908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 204/402] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:34 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:34.738849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf8cdb-2f9b-4c4a-8eb1-1b7c64948791 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.739162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf8cdb-2f9b-4c4a-8eb1-1b7c64948791 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.739443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf8cdb-2f9b-4c4a-8eb1-1b7c64948791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.757613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 199/403] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:34.758736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619be01-03ca-4e93-9eb5-a8ff10ebc7bf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:34.769087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8daf8cdb-2f9b-4c4a-8eb1-1b7c64948791 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:34.770483 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8daf8cdb-2f9b-4c4a-8eb1-1b7c64948791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:34.809443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:34.886512 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619be01-03ca-4e93-9eb5-a8ff10ebc7bf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:34.912789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 205/404] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:34.931279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b1f61-f76e-4807-87d7-75a5a4f63812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:34.932330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b1f61-f76e-4807-87d7-75a5a4f63812 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:34.932748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b1f61-f76e-4807-87d7-75a5a4f63812 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:34.933174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b1f61-f76e-4807-87d7-75a5a4f63812 None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a/groups/ae53619a7e634d7caf14b7d3511f43ed/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:34.935282 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b1f61-f76e-4807-87d7-75a5a4f63812 None admin] RBAC: Authorizing `identity:create_grant(project_id=02dd689032dc41219d425e2f5ccadc7a, group_id=ae53619a7e634d7caf14b7d3511f43ed, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:34.936944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b1f61-f76e-4807-87d7-75a5a4f63812 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:34.953773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 200/405] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 9 13:01:34 2022] PUT /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a/groups/ae53619a7e634d7caf14b7d3511f43ed/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:01:35.079834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf8cdb-2f9b-4c4a-8eb1-1b7c64948791 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:35.220983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b1f61-f76e-4807-87d7-75a5a4f63812 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:35.229078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf8cdb-2f9b-4c4a-8eb1-1b7c64948791 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:35.255199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:35.294470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838d3e1-b291-4757-b54b-8f91b5be80eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:35.295254 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d3e1-b291-4757-b54b-8f91b5be80eb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:35.295475 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d3e1-b291-4757-b54b-8f91b5be80eb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:35.295736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d3e1-b291-4757-b54b-8f91b5be80eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:35.316188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838d3e1-b291-4757-b54b-8f91b5be80eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7838d3e1-b291-4757-b54b-8f91b5be80eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:35.321422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:35.322529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:35.323572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:35.323572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:35.323572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:35.323572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:35.323572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:35.323572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:35.323572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:35.323572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:35.323572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:35.323572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:35.323572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:35.323572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:35.330618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 206/406] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:01:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:01:35.338668 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838d3e1-b291-4757-b54b-8f91b5be80eb None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:35.363767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:35.380801 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b1f61-f76e-4807-87d7-75a5a4f63812 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:35.449989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c84df6-0fee-4b52-b812-3b8167efaa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:35.450641 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c84df6-0fee-4b52-b812-3b8167efaa4a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:35.450921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c84df6-0fee-4b52-b812-3b8167efaa4a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:35.451221 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c84df6-0fee-4b52-b812-3b8167efaa4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:35.476331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c84df6-0fee-4b52-b812-3b8167efaa4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a0c84df6-0fee-4b52-b812-3b8167efaa4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:35.481062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:35.482161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:35.483175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:35.483175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:35.483175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:35.483175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:35.483175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:35.483175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:35.483175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:35.483175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:35.483175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:35.483175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:35.483175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:35.483175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:35.483175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 201/407] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:01:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:35.491436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c84df6-0fee-4b52-b812-3b8167efaa4a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:35.539118 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838d3e1-b291-4757-b54b-8f91b5be80eb None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:35.623442 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77956fde-cd26-47e3-bd28-516cbb761e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:35.624086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77956fde-cd26-47e3-bd28-516cbb761e14 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:35.624369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77956fde-cd26-47e3-bd28-516cbb761e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:35.624640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77956fde-cd26-47e3-bd28-516cbb761e14 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:35.625343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77956fde-cd26-47e3-bd28-516cbb761e14 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:35.625724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77956fde-cd26-47e3-bd28-516cbb761e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:35.627197 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77956fde-cd26-47e3-bd28-516cbb761e14 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:35.639007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 207/408] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:35.645702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77956fde-cd26-47e3-bd28-516cbb761e14 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:35.652021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c84df6-0fee-4b52-b812-3b8167efaa4a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:35.691918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6df8d-a55e-419c-bda6-bf30bd654bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:35.692575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6df8d-a55e-419c-bda6-bf30bd654bfc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:35.692855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6df8d-a55e-419c-bda6-bf30bd654bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:35.693183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6df8d-a55e-419c-bda6-bf30bd654bfc None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:35.693863 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6df8d-a55e-419c-bda6-bf30bd654bfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:35.694741 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6df8d-a55e-419c-bda6-bf30bd654bfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:35.697689 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6df8d-a55e-419c-bda6-bf30bd654bfc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:35.705881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 202/409] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:35.711619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6df8d-a55e-419c-bda6-bf30bd654bfc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:35.738249 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:35.747377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:35.848992 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6df8d-a55e-419c-bda6-bf30bd654bfc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:35.883291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77956fde-cd26-47e3-bd28-516cbb761e14 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:35.900709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0382c88-0c12-4a37-94b2-99937ea9df6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:35.901406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0382c88-0c12-4a37-94b2-99937ea9df6e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:35.901695 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0382c88-0c12-4a37-94b2-99937ea9df6e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:35.901972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0382c88-0c12-4a37-94b2-99937ea9df6e None admin] PATH_INFO: `/v3/groups/1c90f2746e2d447a862039de9dd6a014` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:35.903532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0382c88-0c12-4a37-94b2-99937ea9df6e None admin] RBAC: Authorizing `identity:get_group(group_id=1c90f2746e2d447a862039de9dd6a014)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:35.904993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0382c88-0c12-4a37-94b2-99937ea9df6e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:35.906580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 203/410] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:35 2022] GET /identity/v3/groups/1c90f2746e2d447a862039de9dd6a014 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:35.912078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0382c88-0c12-4a37-94b2-99937ea9df6e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:35.941803 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbc8ee-73e4-4530-8c6e-5211619be6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:35.943499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbc8ee-73e4-4530-8c6e-5211619be6bc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:35.943779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbc8ee-73e4-4530-8c6e-5211619be6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:35.945932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbc8ee-73e4-4530-8c6e-5211619be6bc None admin] PATH_INFO: `/v3/groups/ae53619a7e634d7caf14b7d3511f43ed` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:35.947627 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbc8ee-73e4-4530-8c6e-5211619be6bc None admin] RBAC: Authorizing `identity:get_group(group_id=ae53619a7e634d7caf14b7d3511f43ed)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:35.949327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbc8ee-73e4-4530-8c6e-5211619be6bc None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:35.956100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 208/411] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:35 2022] GET /identity/v3/groups/ae53619a7e634d7caf14b7d3511f43ed => generated 226 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:35.957829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbc8ee-73e4-4530-8c6e-5211619be6bc None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:36.093592 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:36.117747 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0382c88-0c12-4a37-94b2-99937ea9df6e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:36.161280 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbc8ee-73e4-4530-8c6e-5211619be6bc None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:36.174458 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:36.205130 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05dadc0-46b8-4b01-98eb-a68ac14c34fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:36.205950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dadc0-46b8-4b01-98eb-a68ac14c34fe None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:36.206338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dadc0-46b8-4b01-98eb-a68ac14c34fe None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:36.206721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dadc0-46b8-4b01-98eb-a68ac14c34fe None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:36.207546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d764f-4de3-4b1a-8110-fae37d1a9b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:36.208100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05dadc0-46b8-4b01-98eb-a68ac14c34fe None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:36.208531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d764f-4de3-4b1a-8110-fae37d1a9b4d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:36.208967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d764f-4de3-4b1a-8110-fae37d1a9b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:36.209386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d764f-4de3-4b1a-8110-fae37d1a9b4d None admin] PATH_INFO: `/v3/projects/97408104e346493ebafc4d549d2257c4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:36.210459 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05dadc0-46b8-4b01-98eb-a68ac14c34fe None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:36.212536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 209/412] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:35 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:36.213410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d764f-4de3-4b1a-8110-fae37d1a9b4d None admin] RBAC: Authorizing `identity:get_project(project_id=97408104e346493ebafc4d549d2257c4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:36.214863 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d764f-4de3-4b1a-8110-fae37d1a9b4d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:36.221521 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 204/413] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:35 2022] GET /identity/v3/projects/97408104e346493ebafc4d549d2257c4 => generated 344 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:36.222070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05dadc0-46b8-4b01-98eb-a68ac14c34fe None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:36.222737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d764f-4de3-4b1a-8110-fae37d1a9b4d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:36.391179 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05dadc0-46b8-4b01-98eb-a68ac14c34fe None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:36.413263 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d764f-4de3-4b1a-8110-fae37d1a9b4d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:36.456774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f3fd0-5322-4e83-8c24-82150ce0d572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:36.457434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f3fd0-5322-4e83-8c24-82150ce0d572 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:36.457712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f3fd0-5322-4e83-8c24-82150ce0d572 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:36.457984 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f3fd0-5322-4e83-8c24-82150ce0d572 None admin] PATH_INFO: `/v3/projects/97408104e346493ebafc4d549d2257c4/groups/1c90f2746e2d447a862039de9dd6a014/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:36.460186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f3fd0-5322-4e83-8c24-82150ce0d572 None admin] RBAC: Authorizing `identity:create_grant(project_id=97408104e346493ebafc4d549d2257c4, group_id=1c90f2746e2d447a862039de9dd6a014, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:36.461724 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f3fd0-5322-4e83-8c24-82150ce0d572 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:36.466809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686097f0-cbd8-42f9-9b5f-e3c5b6379c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:36.467651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686097f0-cbd8-42f9-9b5f-e3c5b6379c09 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:36.467983 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686097f0-cbd8-42f9-9b5f-e3c5b6379c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:36.468260 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686097f0-cbd8-42f9-9b5f-e3c5b6379c09 None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a/groups/ae53619a7e634d7caf14b7d3511f43ed/roles/0238ab71fb17471cb87e84097272c752` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:36.470538 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686097f0-cbd8-42f9-9b5f-e3c5b6379c09 None admin] RBAC: Authorizing `identity:create_grant(project_id=02dd689032dc41219d425e2f5ccadc7a, group_id=ae53619a7e634d7caf14b7d3511f43ed, role_id=0238ab71fb17471cb87e84097272c752)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:36.472236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686097f0-cbd8-42f9-9b5f-e3c5b6379c09 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:36.484719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 205/414] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 9 13:01:36 2022] PUT /identity/v3/projects/97408104e346493ebafc4d549d2257c4/groups/1c90f2746e2d447a862039de9dd6a014/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:01:36.496889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 210/415] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 9 13:01:36 2022] PUT /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a/groups/ae53619a7e634d7caf14b7d3511f43ed/roles/0238ab71fb17471cb87e84097272c752 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:01:37.069325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb94ba-63eb-4d43-918b-fb9a9e792d22 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:37.069824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb94ba-63eb-4d43-918b-fb9a9e792d22 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:37.070240 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb94ba-63eb-4d43-918b-fb9a9e792d22 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:37.071734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 206/416] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:37.077375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387e899-04d5-4e44-bb5e-b0b769008673 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:37.078234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387e899-04d5-4e44-bb5e-b0b769008673 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:37.078722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387e899-04d5-4e44-bb5e-b0b769008673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:37.108094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b387e899-04d5-4e44-bb5e-b0b769008673 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:37.109676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b387e899-04d5-4e44-bb5e-b0b769008673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:37.147637 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:37.208025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 211/417] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:01:37.272942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751de1ce-43d5-49a5-96c4-f29aee928972 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:37.273895 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751de1ce-43d5-49a5-96c4-f29aee928972 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:37.274419 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751de1ce-43d5-49a5-96c4-f29aee928972 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:37.276285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 207/418] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:37.282050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24d6dd-735c-4e5a-b790-3005155bacdb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:37.282597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24d6dd-735c-4e5a-b790-3005155bacdb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:37.283110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24d6dd-735c-4e5a-b790-3005155bacdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:37.302570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0a24d6dd-735c-4e5a-b790-3005155bacdb None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:37.304186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0a24d6dd-735c-4e5a-b790-3005155bacdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:37.377703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 212/419] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:37.658873 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751de1ce-43d5-49a5-96c4-f29aee928972 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:37.681058 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:37.729047 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24d6dd-735c-4e5a-b790-3005155bacdb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:37.803740 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751de1ce-43d5-49a5-96c4-f29aee928972 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:37.851989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7753d61-9fda-4fcb-b97d-ef88fb6e23aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:37.852739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7753d61-9fda-4fcb-b97d-ef88fb6e23aa None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:37.853091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7753d61-9fda-4fcb-b97d-ef88fb6e23aa None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:37.853431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7753d61-9fda-4fcb-b97d-ef88fb6e23aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:37.854269 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:37.863651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24d6dd-735c-4e5a-b790-3005155bacdb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:37.867254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7753d61-9fda-4fcb-b97d-ef88fb6e23aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e7753d61-9fda-4fcb-b97d-ef88fb6e23aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:37.872650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:37.873585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:37.874378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:37.874378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:37.874378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:37.874378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:37.874378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:37.874378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:37.874378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:37.874378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:37.874378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:37.874378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:37.874378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:37.874378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:37.874378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 208/420] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:37.875028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a659a65-d261-483b-a9e1-9ec82bced360 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:37.879793 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a659a65-d261-483b-a9e1-9ec82bced360 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:37.880091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a659a65-d261-483b-a9e1-9ec82bced360 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:37.882394 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 209/421] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:37.883149 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a659a65-d261-483b-a9e1-9ec82bced360 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:37.911821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0252a5-ca7b-484e-9f60-86e460aea65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:37.913105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0252a5-ca7b-484e-9f60-86e460aea65e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:37.913544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0252a5-ca7b-484e-9f60-86e460aea65e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:37.913920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0252a5-ca7b-484e-9f60-86e460aea65e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:37.925182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0252a5-ca7b-484e-9f60-86e460aea65e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-5b0252a5-ca7b-484e-9f60-86e460aea65e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:37.930508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:37.931733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:37.932778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:37.932778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:37.932778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:37.932778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:37.932778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:37.932778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:37.932778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:37.932778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:37.932778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:37.932778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:37.932778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:37.932778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:37.938296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 213/422] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:01:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:37.938296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8238a0-615f-4824-b827-b94dde3d48b5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:37.938296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8238a0-615f-4824-b827-b94dde3d48b5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:37.938296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8238a0-615f-4824-b827-b94dde3d48b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:37.956570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ed8238a0-615f-4824-b827-b94dde3d48b5 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:37.957785 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ed8238a0-615f-4824-b827-b94dde3d48b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:37.975622 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:37.994909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a659a65-d261-483b-a9e1-9ec82bced360 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:38.030921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c391c-43df-45fd-9ea2-38aecccf8a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:38.031687 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c391c-43df-45fd-9ea2-38aecccf8a8e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:38.031965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c391c-43df-45fd-9ea2-38aecccf8a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:38.032237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c391c-43df-45fd-9ea2-38aecccf8a8e None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:38.032906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c391c-43df-45fd-9ea2-38aecccf8a8e None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:38.033217 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c391c-43df-45fd-9ea2-38aecccf8a8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:38.034583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c391c-43df-45fd-9ea2-38aecccf8a8e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:38.035198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 214/423] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:38.036094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8238a0-615f-4824-b827-b94dde3d48b5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:38.044713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 210/424] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:38.053049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c391c-43df-45fd-9ea2-38aecccf8a8e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:38.206261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:38.232959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8238a0-615f-4824-b827-b94dde3d48b5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:38.273460 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d26e6-1f35-426f-b07a-788400268b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:38.274087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d26e6-1f35-426f-b07a-788400268b3f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:38.274366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d26e6-1f35-426f-b07a-788400268b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:38.274640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d26e6-1f35-426f-b07a-788400268b3f None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:38.274966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c391c-43df-45fd-9ea2-38aecccf8a8e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:38.275411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d26e6-1f35-426f-b07a-788400268b3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:38.275640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d26e6-1f35-426f-b07a-788400268b3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:38.277018 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d26e6-1f35-426f-b07a-788400268b3f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:38.288144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 215/425] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:38.294839 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d26e6-1f35-426f-b07a-788400268b3f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:38.364469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad02c6a-d3cf-4ef9-ae78-8238be4bbd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:38.364828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad02c6a-d3cf-4ef9-ae78-8238be4bbd0f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:38.365110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad02c6a-d3cf-4ef9-ae78-8238be4bbd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:38.365406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad02c6a-d3cf-4ef9-ae78-8238be4bbd0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:38.387994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:38.416694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d26e6-1f35-426f-b07a-788400268b3f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:38.453626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad02c6a-d3cf-4ef9-ae78-8238be4bbd0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:38.455110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad02c6a-d3cf-4ef9-ae78-8238be4bbd0f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:38.461570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da58c0-e0f7-49d3-8d08-9c715f6aca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:38.462401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da58c0-e0f7-49d3-8d08-9c715f6aca81 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:38.462859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da58c0-e0f7-49d3-8d08-9c715f6aca81 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:38.463332 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da58c0-e0f7-49d3-8d08-9c715f6aca81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:38.482014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:38.551678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da58c0-e0f7-49d3-8d08-9c715f6aca81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:38.554487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da58c0-e0f7-49d3-8d08-9c715f6aca81 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:38.578643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 211/426] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 13:01:38.579650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad02c6a-d3cf-4ef9-ae78-8238be4bbd0f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:38.639712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:38.682135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 216/427] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:38.682866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da58c0-e0f7-49d3-8d08-9c715f6aca81 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:38.709612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad02c6a-d3cf-4ef9-ae78-8238be4bbd0f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:38.732792 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:38.753009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b2372-3a09-48ae-a3c9-e56bda8551cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:38.754120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b2372-3a09-48ae-a3c9-e56bda8551cf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:38.754602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b2372-3a09-48ae-a3c9-e56bda8551cf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:38.755112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b2372-3a09-48ae-a3c9-e56bda8551cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:38.766976 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b2372-3a09-48ae-a3c9-e56bda8551cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-378b2372-3a09-48ae-a3c9-e56bda8551cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:38.772586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:38.773498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:38.774329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:38.774329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:38.774329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:38.774329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:38.774329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:38.774329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:38.774329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:38.774329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:38.774329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:38.774329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:38.774329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 13:01:38.774329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:38.777116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 212/428] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:38.778196 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b2372-3a09-48ae-a3c9-e56bda8551cf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:38.796983 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da58c0-e0f7-49d3-8d08-9c715f6aca81 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:38.837052 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7ba8e-9686-4e9f-9b02-7c37f94c6706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:38.837685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7ba8e-9686-4e9f-9b02-7c37f94c6706 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:38.837959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7ba8e-9686-4e9f-9b02-7c37f94c6706 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:38.838228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7ba8e-9686-4e9f-9b02-7c37f94c6706 None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:38.839521 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7ba8e-9686-4e9f-9b02-7c37f94c6706 None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:38.840943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7ba8e-9686-4e9f-9b02-7c37f94c6706 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:38.842859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 217/429] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:38 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:38.843650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7ba8e-9686-4e9f-9b02-7c37f94c6706 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:38.893259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:38.899786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b2372-3a09-48ae-a3c9-e56bda8551cf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:38.940098 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fda943-8673-420f-8ee8-f3e7e82902f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:38.940736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda943-8673-420f-8ee8-f3e7e82902f2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:38.941266 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda943-8673-420f-8ee8-f3e7e82902f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:38.941543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda943-8673-420f-8ee8-f3e7e82902f2 None admin] PATH_INFO: `/v3/projects/482e7368e31f415faa7b8b65d04ae50f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:38.942681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fda943-8673-420f-8ee8-f3e7e82902f2 None admin] RBAC: Authorizing `identity:get_project(project_id=482e7368e31f415faa7b8b65d04ae50f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:38.944798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fda943-8673-420f-8ee8-f3e7e82902f2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:38.946490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 213/430] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:38 2022] GET /identity/v3/projects/482e7368e31f415faa7b8b65d04ae50f => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:38.947269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fda943-8673-420f-8ee8-f3e7e82902f2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:38.960605 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7ba8e-9686-4e9f-9b02-7c37f94c6706 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:38.977134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:39.000120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539677cf-9795-463b-b25e-d9ed713e1e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:39.000787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539677cf-9795-463b-b25e-d9ed713e1e14 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.001069 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539677cf-9795-463b-b25e-d9ed713e1e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.001340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539677cf-9795-463b-b25e-d9ed713e1e14 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.002009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539677cf-9795-463b-b25e-d9ed713e1e14 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:39.002310 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539677cf-9795-463b-b25e-d9ed713e1e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:39.003817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539677cf-9795-463b-b25e-d9ed713e1e14 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:39.018193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 218/431] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:01:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:39.019249 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912d688-bac3-4de1-8b81-6f9646ccf9ae None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.019538 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912d688-bac3-4de1-8b81-6f9646ccf9ae None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.019814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912d688-bac3-4de1-8b81-6f9646ccf9ae None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.023234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 219/432] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:39.023234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912d688-bac3-4de1-8b81-6f9646ccf9ae None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:39.056763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fda943-8673-420f-8ee8-f3e7e82902f2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:39.097897 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25657003-2536-454e-a845-fda0089d1884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:39.098554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25657003-2536-454e-a845-fda0089d1884 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.098832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25657003-2536-454e-a845-fda0089d1884 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.099128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25657003-2536-454e-a845-fda0089d1884 None admin] PATH_INFO: `/v3/groups/ae53619a7e634d7caf14b7d3511f43ed` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.100450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25657003-2536-454e-a845-fda0089d1884 None admin] RBAC: Authorizing `identity:get_group(group_id=ae53619a7e634d7caf14b7d3511f43ed)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:39.102145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25657003-2536-454e-a845-fda0089d1884 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:39.104060 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 214/433] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:38 2022] GET /identity/v3/groups/ae53619a7e634d7caf14b7d3511f43ed => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:39.104945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25657003-2536-454e-a845-fda0089d1884 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:39.141828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:39.144476 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912d688-bac3-4de1-8b81-6f9646ccf9ae None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:39.180596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5af047-eb4e-4202-9020-a0d03d072a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:39.181228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5af047-eb4e-4202-9020-a0d03d072a44 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.181505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5af047-eb4e-4202-9020-a0d03d072a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.181772 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5af047-eb4e-4202-9020-a0d03d072a44 None admin] PATH_INFO: `/v3/groups/ae53619a7e634d7caf14b7d3511f43ed` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.182885 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5af047-eb4e-4202-9020-a0d03d072a44 None admin] RBAC: Authorizing `identity:get_group(group_id=ae53619a7e634d7caf14b7d3511f43ed)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:39.184338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5af047-eb4e-4202-9020-a0d03d072a44 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:39.185904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 220/434] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:39 2022] GET /identity/v3/groups/ae53619a7e634d7caf14b7d3511f43ed => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:39.186994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84942dd-736d-42b5-bd5e-5c7d2a9e9fa6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.187294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84942dd-736d-42b5-bd5e-5c7d2a9e9fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.187562 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84942dd-736d-42b5-bd5e-5c7d2a9e9fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.205665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d84942dd-736d-42b5-bd5e-5c7d2a9e9fa6 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:39.206872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d84942dd-736d-42b5-bd5e-5c7d2a9e9fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:39.217658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25657003-2536-454e-a845-fda0089d1884 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:39.247523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:39.251993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5100130-9d23-475a-a8ce-54fa725740be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:39.252539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5100130-9d23-475a-a8ce-54fa725740be None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.252811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5100130-9d23-475a-a8ce-54fa725740be None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.253084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5100130-9d23-475a-a8ce-54fa725740be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.275529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 221/435] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:39.276366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84942dd-736d-42b5-bd5e-5c7d2a9e9fa6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:39.339509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5100130-9d23-475a-a8ce-54fa725740be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:39.340969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5100130-9d23-475a-a8ce-54fa725740be None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:39.375327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:39.380148 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84942dd-736d-42b5-bd5e-5c7d2a9e9fa6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:39.416721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481b7f0-e1c6-4cc7-9fa2-ef32c89974ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:39.417334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481b7f0-e1c6-4cc7-9fa2-ef32c89974ad None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.417613 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481b7f0-e1c6-4cc7-9fa2-ef32c89974ad None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.417883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481b7f0-e1c6-4cc7-9fa2-ef32c89974ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.418607 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481b7f0-e1c6-4cc7-9fa2-ef32c89974ad None admin] RBAC: Adding query filter params (group.id=ae53619a7e634d7caf14b7d3511f43ed, role.id=e7b95eb3b5ad4479b50aaa68e58591ea, scope.project.id=482e7368e31f415faa7b8b65d04ae50f) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:39.418917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481b7f0-e1c6-4cc7-9fa2-ef32c89974ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:39.420417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481b7f0-e1c6-4cc7-9fa2-ef32c89974ad None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:39.426697 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 222/436] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 13:01:39 2022] GET /identity/v3/role_assignments?group.id=ae53619a7e634d7caf14b7d3511f43ed&scope.project.id=482e7368e31f415faa7b8b65d04ae50f&role.id=e7b95eb3b5ad4479b50aaa68e58591ea => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:39.427943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920f205-54a6-4382-9758-a14d470b43e4 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.428227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920f205-54a6-4382-9758-a14d470b43e4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.428532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920f205-54a6-4382-9758-a14d470b43e4 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.434001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 223/437] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:39.434001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920f205-54a6-4382-9758-a14d470b43e4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:39.463009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 215/438] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:39.464652 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fa5f6-c93f-4726-9150-e4d43e598ff8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.465147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fa5f6-c93f-4726-9150-e4d43e598ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.465610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fa5f6-c93f-4726-9150-e4d43e598ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.483445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:39.486701 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fc9fa5f6-c93f-4726-9150-e4d43e598ff8 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:39.490121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fc9fa5f6-c93f-4726-9150-e4d43e598ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:39.554218 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920f205-54a6-4382-9758-a14d470b43e4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:39.567561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 216/439] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:39.576061 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fa5f6-c93f-4726-9150-e4d43e598ff8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:39.599593 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8aba0-8daa-4b40-bf4e-6e5ce2a77f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:39.600832 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8aba0-8daa-4b40-bf4e-6e5ce2a77f42 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.601335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8aba0-8daa-4b40-bf4e-6e5ce2a77f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.601808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8aba0-8daa-4b40-bf4e-6e5ce2a77f42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.602794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8aba0-8daa-4b40-bf4e-6e5ce2a77f42 None admin] RBAC: Adding query filter params (group.id=ae53619a7e634d7caf14b7d3511f43ed, role.id=0238ab71fb17471cb87e84097272c752, scope.project.id=482e7368e31f415faa7b8b65d04ae50f) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:39.603361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8aba0-8daa-4b40-bf4e-6e5ce2a77f42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:39.605112 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8aba0-8daa-4b40-bf4e-6e5ce2a77f42 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:39.615910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 224/440] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 13:01:39 2022] GET /identity/v3/role_assignments?group.id=ae53619a7e634d7caf14b7d3511f43ed&scope.project.id=482e7368e31f415faa7b8b65d04ae50f&role.id=0238ab71fb17471cb87e84097272c752 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:39.615910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8aba0-8daa-4b40-bf4e-6e5ce2a77f42 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:39.674729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:39.729028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fa5f6-c93f-4726-9150-e4d43e598ff8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:39.756169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8aba0-8daa-4b40-bf4e-6e5ce2a77f42 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:39.758135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:39.778570 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fb1dd-5c4a-40ff-acc2-005f259928f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:39.779288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fb1dd-5c4a-40ff-acc2-005f259928f0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.780157 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fb1dd-5c4a-40ff-acc2-005f259928f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.780432 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fb1dd-5c4a-40ff-acc2-005f259928f0 None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.781909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fb1dd-5c4a-40ff-acc2-005f259928f0 None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:39.784121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fb1dd-5c4a-40ff-acc2-005f259928f0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:39.786189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 217/441] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:39 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:39.792356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fb1dd-5c4a-40ff-acc2-005f259928f0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:39.811894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078120b-37f0-40a5-940e-ecbf88df87cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:39.812468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078120b-37f0-40a5-940e-ecbf88df87cc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.812748 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078120b-37f0-40a5-940e-ecbf88df87cc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.813020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078120b-37f0-40a5-940e-ecbf88df87cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.826635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078120b-37f0-40a5-940e-ecbf88df87cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a078120b-37f0-40a5-940e-ecbf88df87cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:39.832865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:39.833774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:39.835207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:39.835207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:39.835207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:39.835207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:39.835207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:39.835207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:39.835207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:39.835207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:39.835207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:39.835207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:39.835207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:39.835207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:39.835207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 225/442] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:01:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:39.840664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078120b-37f0-40a5-940e-ecbf88df87cc None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:39.922925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fb1dd-5c4a-40ff-acc2-005f259928f0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:39.960030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:39.968265 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de63127-db2a-49b8-a0dc-06c0e407d7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:39.968909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de63127-db2a-49b8-a0dc-06c0e407d7fc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.969523 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de63127-db2a-49b8-a0dc-06c0e407d7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.969807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de63127-db2a-49b8-a0dc-06c0e407d7fc None admin] PATH_INFO: `/v3/groups/ae53619a7e634d7caf14b7d3511f43ed` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.973309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de63127-db2a-49b8-a0dc-06c0e407d7fc None admin] RBAC: Authorizing `identity:get_group(group_id=ae53619a7e634d7caf14b7d3511f43ed)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:39.973309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de63127-db2a-49b8-a0dc-06c0e407d7fc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:39.974295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 218/443] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:39 2022] GET /identity/v3/groups/ae53619a7e634d7caf14b7d3511f43ed => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:39.975465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de63127-db2a-49b8-a0dc-06c0e407d7fc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:39.977812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078120b-37f0-40a5-940e-ecbf88df87cc None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:40.015805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9ba03-2705-4e30-881b-f0d703fca08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:40.016799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9ba03-2705-4e30-881b-f0d703fca08a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:40.016799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9ba03-2705-4e30-881b-f0d703fca08a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:40.016965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9ba03-2705-4e30-881b-f0d703fca08a None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:40.017617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9ba03-2705-4e30-881b-f0d703fca08a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:40.017921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9ba03-2705-4e30-881b-f0d703fca08a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:40.019341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9ba03-2705-4e30-881b-f0d703fca08a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:40.031355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 226/444] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:40.032108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9ba03-2705-4e30-881b-f0d703fca08a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:40.060627 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:40.100562 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de63127-db2a-49b8-a0dc-06c0e407d7fc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:40.140550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a31dfe-caa1-4dd0-8fa5-59f1e92e79db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:40.141188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a31dfe-caa1-4dd0-8fa5-59f1e92e79db None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:40.141465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a31dfe-caa1-4dd0-8fa5-59f1e92e79db None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:40.141738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a31dfe-caa1-4dd0-8fa5-59f1e92e79db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:40.153814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a31dfe-caa1-4dd0-8fa5-59f1e92e79db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:40.157204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9ba03-2705-4e30-881b-f0d703fca08a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-19a31dfe-caa1-4dd0-8fa5-59f1e92e79db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:40.158989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:40.160008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:40.160810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:40.160810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:40.160810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:40.160810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:40.160810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:40.160810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:40.160810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:40.160810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:40.160810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:40.160810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:40.160810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:01:40.160810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:40.160810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 219/445] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:01:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:40.161518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a31dfe-caa1-4dd0-8fa5-59f1e92e79db None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:40.198650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bcd57-8c0b-4a70-8347-d307745ade2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:40.199402 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bcd57-8c0b-4a70-8347-d307745ade2c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:40.199682 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bcd57-8c0b-4a70-8347-d307745ade2c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:40.199946 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bcd57-8c0b-4a70-8347-d307745ade2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:40.200665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bcd57-8c0b-4a70-8347-d307745ade2c None admin] RBAC: Adding query filter params (group.id=ae53619a7e634d7caf14b7d3511f43ed, role.id=e7b95eb3b5ad4479b50aaa68e58591ea, scope.project.id=02dd689032dc41219d425e2f5ccadc7a) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:40.200963 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bcd57-8c0b-4a70-8347-d307745ade2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:40.202398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bcd57-8c0b-4a70-8347-d307745ade2c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:40.208532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 227/446] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 13:01:40 2022] GET /identity/v3/role_assignments?group.id=ae53619a7e634d7caf14b7d3511f43ed&scope.project.id=02dd689032dc41219d425e2f5ccadc7a&role.id=e7b95eb3b5ad4479b50aaa68e58591ea => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:40.209280 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bcd57-8c0b-4a70-8347-d307745ade2c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:40.231510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:40.279349 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a31dfe-caa1-4dd0-8fa5-59f1e92e79db None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:40.308173 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:40.320609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484eeda-2939-4146-b807-0c0de6ba0047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:40.321275 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484eeda-2939-4146-b807-0c0de6ba0047 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:40.321515 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484eeda-2939-4146-b807-0c0de6ba0047 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:40.321871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484eeda-2939-4146-b807-0c0de6ba0047 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:40.324207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bcd57-8c0b-4a70-8347-d307745ade2c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:40.364495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6155b4-ecbd-4cc9-9ec7-9b6f42b7da4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:40.365071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6155b4-ecbd-4cc9-9ec7-9b6f42b7da4a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:40.365400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6155b4-ecbd-4cc9-9ec7-9b6f42b7da4a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:40.365680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6155b4-ecbd-4cc9-9ec7-9b6f42b7da4a None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:40.366448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6155b4-ecbd-4cc9-9ec7-9b6f42b7da4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:40.366816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6155b4-ecbd-4cc9-9ec7-9b6f42b7da4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:40.368290 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6155b4-ecbd-4cc9-9ec7-9b6f42b7da4a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:40.378094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 228/447] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:40.384992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6155b4-ecbd-4cc9-9ec7-9b6f42b7da4a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:40.399917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b484eeda-2939-4146-b807-0c0de6ba0047 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:40.402539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b484eeda-2939-4146-b807-0c0de6ba0047 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:40.470816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:40.499201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6155b4-ecbd-4cc9-9ec7-9b6f42b7da4a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:40.524490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 220/448] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:40.530921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484eeda-2939-4146-b807-0c0de6ba0047 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:40.539811 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fe15b-6e24-47d1-aa92-45e24a4e69d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:40.540441 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fe15b-6e24-47d1-aa92-45e24a4e69d4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:40.540722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fe15b-6e24-47d1-aa92-45e24a4e69d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:40.540994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fe15b-6e24-47d1-aa92-45e24a4e69d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:40.563666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:40.625255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fe15b-6e24-47d1-aa92-45e24a4e69d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:40.626631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fe15b-6e24-47d1-aa92-45e24a4e69d4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:40.651775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484eeda-2939-4146-b807-0c0de6ba0047 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:40.685259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b172660-913e-4092-a85e-29389112a105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:40.685882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b172660-913e-4092-a85e-29389112a105 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:40.686159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b172660-913e-4092-a85e-29389112a105 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:40.686439 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b172660-913e-4092-a85e-29389112a105 None admin] PATH_INFO: `/v3/projects/02dd689032dc41219d425e2f5ccadc7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:40.687583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b172660-913e-4092-a85e-29389112a105 None admin] RBAC: Authorizing `identity:get_project(project_id=02dd689032dc41219d425e2f5ccadc7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:40.689023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b172660-913e-4092-a85e-29389112a105 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:40.690899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 221/449] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:40 2022] GET /identity/v3/projects/02dd689032dc41219d425e2f5ccadc7a => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:40.697011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b172660-913e-4092-a85e-29389112a105 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:40.723632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:40.724150 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 229/450] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 13:01:40.729799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fe15b-6e24-47d1-aa92-45e24a4e69d4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:40.778589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:40.797109 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b172660-913e-4092-a85e-29389112a105 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:40.828900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fe15b-6e24-47d1-aa92-45e24a4e69d4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:40.831497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464b2a2-8cb5-46b6-83b1-7f4f52244761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:40.832126 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464b2a2-8cb5-46b6-83b1-7f4f52244761 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:40.832405 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464b2a2-8cb5-46b6-83b1-7f4f52244761 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:40.832679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464b2a2-8cb5-46b6-83b1-7f4f52244761 None admin] PATH_INFO: `/v3/groups/ae53619a7e634d7caf14b7d3511f43ed` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:40.834323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464b2a2-8cb5-46b6-83b1-7f4f52244761 None admin] RBAC: Authorizing `identity:get_group(group_id=ae53619a7e634d7caf14b7d3511f43ed)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:40.836076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464b2a2-8cb5-46b6-83b1-7f4f52244761 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:40.837661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 222/451] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:40 2022] GET /identity/v3/groups/ae53619a7e634d7caf14b7d3511f43ed => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:40.843357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464b2a2-8cb5-46b6-83b1-7f4f52244761 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:40.862830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4938b1-1f7d-4b0a-b2e3-d2352e7e002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:40.863494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4938b1-1f7d-4b0a-b2e3-d2352e7e002b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:40.863778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4938b1-1f7d-4b0a-b2e3-d2352e7e002b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:40.864053 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4938b1-1f7d-4b0a-b2e3-d2352e7e002b None admin] PATH_INFO: `/v3/projects/97408104e346493ebafc4d549d2257c4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:40.865207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4938b1-1f7d-4b0a-b2e3-d2352e7e002b None admin] RBAC: Authorizing `identity:get_project(project_id=97408104e346493ebafc4d549d2257c4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:40.866633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4938b1-1f7d-4b0a-b2e3-d2352e7e002b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:40.868297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 230/452] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:01:40 2022] GET /identity/v3/projects/97408104e346493ebafc4d549d2257c4 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:40.873717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4938b1-1f7d-4b0a-b2e3-d2352e7e002b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:40.937331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:40.944816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464b2a2-8cb5-46b6-83b1-7f4f52244761 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:40.972609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4938b1-1f7d-4b0a-b2e3-d2352e7e002b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:40.979027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdc081-a2ea-4a3b-a5c4-21537f0450df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:40.979656 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdc081-a2ea-4a3b-a5c4-21537f0450df None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:40.979932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdc081-a2ea-4a3b-a5c4-21537f0450df None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:40.980202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdc081-a2ea-4a3b-a5c4-21537f0450df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:40.980925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdc081-a2ea-4a3b-a5c4-21537f0450df None admin] RBAC: Adding query filter params (group.id=ae53619a7e634d7caf14b7d3511f43ed, role.id=0238ab71fb17471cb87e84097272c752, scope.project.id=02dd689032dc41219d425e2f5ccadc7a) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:40.981226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdc081-a2ea-4a3b-a5c4-21537f0450df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:40.982656 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdc081-a2ea-4a3b-a5c4-21537f0450df None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:40.988309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 223/453] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 13:01:40 2022] GET /identity/v3/role_assignments?group.id=ae53619a7e634d7caf14b7d3511f43ed&scope.project.id=02dd689032dc41219d425e2f5ccadc7a&role.id=0238ab71fb17471cb87e84097272c752 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:40.990649 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:41.009760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa8857-9672-436b-af00-7fd911a58b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:41.010420 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa8857-9672-436b-af00-7fd911a58b30 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:41.010701 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa8857-9672-436b-af00-7fd911a58b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:41.010979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa8857-9672-436b-af00-7fd911a58b30 None admin] PATH_INFO: `/v3/groups/1c90f2746e2d447a862039de9dd6a014` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:41.012218 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa8857-9672-436b-af00-7fd911a58b30 None admin] RBAC: Authorizing `identity:get_group(group_id=1c90f2746e2d447a862039de9dd6a014)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:41.013691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa8857-9672-436b-af00-7fd911a58b30 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:41.015622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 231/454] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 9 13:01:40 2022] GET /identity/v3/groups/1c90f2746e2d447a862039de9dd6a014 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:01:41.022257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdc081-a2ea-4a3b-a5c4-21537f0450df None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:41.132213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdc081-a2ea-4a3b-a5c4-21537f0450df None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:41.172216 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a95b3-d6d3-4cb1-a2f3-2dfe9f59e26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:41.172825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a95b3-d6d3-4cb1-a2f3-2dfe9f59e26f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:41.173435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a95b3-d6d3-4cb1-a2f3-2dfe9f59e26f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:41.173435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a95b3-d6d3-4cb1-a2f3-2dfe9f59e26f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:41.174454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a95b3-d6d3-4cb1-a2f3-2dfe9f59e26f None admin] RBAC: Adding query filter params (group.id=1c90f2746e2d447a862039de9dd6a014, role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=97408104e346493ebafc4d549d2257c4) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:41.174764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a95b3-d6d3-4cb1-a2f3-2dfe9f59e26f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:41.176421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a95b3-d6d3-4cb1-a2f3-2dfe9f59e26f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:41.185848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 224/455] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 9 13:01:41 2022] GET /identity/v3/role_assignments?group.id=1c90f2746e2d447a862039de9dd6a014&scope.project.id=97408104e346493ebafc4d549d2257c4&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:43.845022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76ad08-8883-4031-bff6-6edea8372d3b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:43.845987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76ad08-8883-4031-bff6-6edea8372d3b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:43.846466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76ad08-8883-4031-bff6-6edea8372d3b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:43.848279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 232/456] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:43.853712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ddb03-4f7c-4c4b-b3e7-ee52adc84d61 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:43.854568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ddb03-4f7c-4c4b-b3e7-ee52adc84d61 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:43.855030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ddb03-4f7c-4c4b-b3e7-ee52adc84d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:43.878720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-293ddb03-4f7c-4c4b-b3e7-ee52adc84d61 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:43.880804 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-293ddb03-4f7c-4c4b-b3e7-ee52adc84d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:43.907710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:43.933635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd6690-60cd-41df-be6a-0d6157917fc9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:43.935196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd6690-60cd-41df-be6a-0d6157917fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:43.935692 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd6690-60cd-41df-be6a-0d6157917fc9 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:43.937849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 233/457] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:01:43.946570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252e5ee-7e49-463e-b990-736621311cf6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:43.947178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252e5ee-7e49-463e-b990-736621311cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:43.947662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252e5ee-7e49-463e-b990-736621311cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:43.978944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a252e5ee-7e49-463e-b990-736621311cf6 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:43.980198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a252e5ee-7e49-463e-b990-736621311cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:43.991137 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 225/458] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:44.089372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 234/459] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:01:44.204787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da402b-f48a-4079-8eb0-0880d9ae1e42 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:44.205120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da402b-f48a-4079-8eb0-0880d9ae1e42 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:44.205339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da402b-f48a-4079-8eb0-0880d9ae1e42 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:44.206685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 226/460] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:44.225222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9888e6c-5518-4e53-bf92-12c1f96597fb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:44.225505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9888e6c-5518-4e53-bf92-12c1f96597fb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:44.225784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9888e6c-5518-4e53-bf92-12c1f96597fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:44.257021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d9888e6c-5518-4e53-bf92-12c1f96597fb None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:44.257601 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d9888e6c-5518-4e53-bf92-12c1f96597fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:44.289026 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da402b-f48a-4079-8eb0-0880d9ae1e42 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:44.373402 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 235/461] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:44.385059 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9888e6c-5518-4e53-bf92-12c1f96597fb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:44.471370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:44.486764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da402b-f48a-4079-8eb0-0880d9ae1e42 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:44.554411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280147c-062b-4709-ac0f-2157684b4ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:44.555077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280147c-062b-4709-ac0f-2157684b4ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:44.555354 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280147c-062b-4709-ac0f-2157684b4ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:44.555636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280147c-062b-4709-ac0f-2157684b4ec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:44.575557 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9888e6c-5518-4e53-bf92-12c1f96597fb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:44.578835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280147c-062b-4709-ac0f-2157684b4ec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:44.580345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280147c-062b-4709-ac0f-2157684b4ec2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5280147c-062b-4709-ac0f-2157684b4ec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:44.606778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:44.609378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:44.611462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:44.611462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:44.611462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:44.611462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 227/462] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:44.623127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280147c-062b-4709-ac0f-2157684b4ec2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:44.636161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:44.654767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fed38b-0a85-4b89-9fdb-90001c1ea603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:44.655436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fed38b-0a85-4b89-9fdb-90001c1ea603 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:44.655720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fed38b-0a85-4b89-9fdb-90001c1ea603 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:44.655995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fed38b-0a85-4b89-9fdb-90001c1ea603 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:44.671039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fed38b-0a85-4b89-9fdb-90001c1ea603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:44.672814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fed38b-0a85-4b89-9fdb-90001c1ea603 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-43fed38b-0a85-4b89-9fdb-90001c1ea603 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:44.693415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:44.694373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:44.695242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:44.695242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:44.695242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:44.695242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 236/463] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:01:44.695711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fed38b-0a85-4b89-9fdb-90001c1ea603 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:44.758663 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280147c-062b-4709-ac0f-2157684b4ec2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:44.794753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c4092-62cc-42d6-aa2e-aeb988ae59e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:44.795902 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c4092-62cc-42d6-aa2e-aeb988ae59e4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:44.796318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c4092-62cc-42d6-aa2e-aeb988ae59e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:44.797101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c4092-62cc-42d6-aa2e-aeb988ae59e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:44.797890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c4092-62cc-42d6-aa2e-aeb988ae59e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:44.798329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c4092-62cc-42d6-aa2e-aeb988ae59e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:44.800388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c4092-62cc-42d6-aa2e-aeb988ae59e4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:44.809878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:44.815565 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fed38b-0a85-4b89-9fdb-90001c1ea603 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:44.818608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 228/464] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:44.820479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c4092-62cc-42d6-aa2e-aeb988ae59e4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:44.860456 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f5036-a22d-4a12-af12-904f837052a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:44.861855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f5036-a22d-4a12-af12-904f837052a6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:44.862362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f5036-a22d-4a12-af12-904f837052a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:44.862834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f5036-a22d-4a12-af12-904f837052a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:44.875849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f5036-a22d-4a12-af12-904f837052a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:44.877676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f5036-a22d-4a12-af12-904f837052a6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-9e7f5036-a22d-4a12-af12-904f837052a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:44.894650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:44.896170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:44.897189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:44.897189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:44.897189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:44.899245 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 237/465] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:44.900234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f5036-a22d-4a12-af12-904f837052a6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:44.910529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:44.953448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c4092-62cc-42d6-aa2e-aeb988ae59e4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:44.992576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73be02-2aa0-4acd-948e-aade16d8c984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:44.993564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73be02-2aa0-4acd-948e-aade16d8c984 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:44.994039 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73be02-2aa0-4acd-948e-aade16d8c984 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:44.994497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73be02-2aa0-4acd-948e-aade16d8c984 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:44.995363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa73be02-2aa0-4acd-948e-aade16d8c984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:44.995671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa73be02-2aa0-4acd-948e-aade16d8c984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:44.997081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa73be02-2aa0-4acd-948e-aade16d8c984 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:45.010755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 229/466] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:45.011773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73be02-2aa0-4acd-948e-aade16d8c984 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:45.033407 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f5036-a22d-4a12-af12-904f837052a6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:45.070567 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f479e-f620-4d15-a97d-eb6f86dfeb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:45.071214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f479e-f620-4d15-a97d-eb6f86dfeb25 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:45.071497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f479e-f620-4d15-a97d-eb6f86dfeb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:45.071767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f479e-f620-4d15-a97d-eb6f86dfeb25 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:45.072420 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f479e-f620-4d15-a97d-eb6f86dfeb25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:45.073841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f479e-f620-4d15-a97d-eb6f86dfeb25 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:45.098720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 238/467] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:01:44 2022] POST /identity/v3/users => generated 246 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:01:45.100012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43532b85-7e96-4501-8159-cdc320b5cfab None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:45.100302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43532b85-7e96-4501-8159-cdc320b5cfab None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:45.100576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43532b85-7e96-4501-8159-cdc320b5cfab None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:45.101942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 239/468] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:45.102494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43532b85-7e96-4501-8159-cdc320b5cfab None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:45.117380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:45.149072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73be02-2aa0-4acd-948e-aade16d8c984 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:45.183450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:45.196841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa04559-1d18-4749-b197-51c4d7ca94de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:45.197991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa04559-1d18-4749-b197-51c4d7ca94de None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:45.198545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa04559-1d18-4749-b197-51c4d7ca94de None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:45.199518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa04559-1d18-4749-b197-51c4d7ca94de None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:45.208628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa04559-1d18-4749-b197-51c4d7ca94de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:45.208628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa04559-1d18-4749-b197-51c4d7ca94de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:45.208628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa04559-1d18-4749-b197-51c4d7ca94de None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:45.222871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 230/469] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:45.224759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a6ee7-74ed-4e65-9635-f678cd531ceb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:45.225048 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a6ee7-74ed-4e65-9635-f678cd531ceb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:45.225320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a6ee7-74ed-4e65-9635-f678cd531ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:45.246767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2b8a6ee7-74ed-4e65-9635-f678cd531ceb None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:45.248142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2b8a6ee7-74ed-4e65-9635-f678cd531ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:45.253296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43532b85-7e96-4501-8159-cdc320b5cfab None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:45.298755 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866ae53-26ac-4694-93f9-101869c41809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:45.300382 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866ae53-26ac-4694-93f9-101869c41809 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:45.300676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866ae53-26ac-4694-93f9-101869c41809 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:45.300947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866ae53-26ac-4694-93f9-101869c41809 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:45.302613 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866ae53-26ac-4694-93f9-101869c41809 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:45.304349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866ae53-26ac-4694-93f9-101869c41809 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:45.329256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 240/470] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:01:45 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:01:45.330971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866ae53-26ac-4694-93f9-101869c41809 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:45.335844 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 231/471] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:45.410880 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:45.429498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7d99d-b1db-469c-9873-bfaabdfa5e88 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:45.430819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7d99d-b1db-469c-9873-bfaabdfa5e88 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:45.431338 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7d99d-b1db-469c-9873-bfaabdfa5e88 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:45.433393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 232/472] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:45.439037 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414c02e-79e0-4234-bb04-251c674cce11 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:45.439655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414c02e-79e0-4234-bb04-251c674cce11 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:45.440423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414c02e-79e0-4234-bb04-251c674cce11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:45.466106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7414c02e-79e0-4234-bb04-251c674cce11 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:45.467400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7414c02e-79e0-4234-bb04-251c674cce11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:45.467881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866ae53-26ac-4694-93f9-101869c41809 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:45.511370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21c0ea-6dfe-4c48-a26c-1b6ebd41a109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:45.512001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c0ea-6dfe-4c48-a26c-1b6ebd41a109 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:45.512288 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c0ea-6dfe-4c48-a26c-1b6ebd41a109 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:45.512560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c0ea-6dfe-4c48-a26c-1b6ebd41a109 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:45.513221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21c0ea-6dfe-4c48-a26c-1b6ebd41a109 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:45.514848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21c0ea-6dfe-4c48-a26c-1b6ebd41a109 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:45.548432 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 241/473] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:01:45 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:01:45.555141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 233/474] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:45.673692 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21c0ea-6dfe-4c48-a26c-1b6ebd41a109 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:45.785556 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21c0ea-6dfe-4c48-a26c-1b6ebd41a109 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:45.805339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:45.826162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef092b0-7390-4619-975e-3f92c2c50842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:45.826804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef092b0-7390-4619-975e-3f92c2c50842 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:45.827938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef092b0-7390-4619-975e-3f92c2c50842 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:45.827938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef092b0-7390-4619-975e-3f92c2c50842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:45.842411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef092b0-7390-4619-975e-3f92c2c50842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:45.844241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef092b0-7390-4619-975e-3f92c2c50842 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:45.847867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414c02e-79e0-4234-bb04-251c674cce11 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6ef092b0-7390-4619-975e-3f92c2c50842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:45.860238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:45.861834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:45.862890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:45.862890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:45.862890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:45.865219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 242/475] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:45.873947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef092b0-7390-4619-975e-3f92c2c50842 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:45.935071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:46.047618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414c02e-79e0-4234-bb04-251c674cce11 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:46.072036 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef092b0-7390-4619-975e-3f92c2c50842 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:46.121178 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59a346-17ad-451d-a533-ab1d3bd7ba74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:46.121803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59a346-17ad-451d-a533-ab1d3bd7ba74 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:46.122082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59a346-17ad-451d-a533-ab1d3bd7ba74 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:46.122356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59a346-17ad-451d-a533-ab1d3bd7ba74 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:46.123066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59a346-17ad-451d-a533-ab1d3bd7ba74 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:46.124466 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59a346-17ad-451d-a533-ab1d3bd7ba74 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:46.127847 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a83cac-fd7e-445c-8a31-707dc844df12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:46.128493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a83cac-fd7e-445c-8a31-707dc844df12 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:46.128778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a83cac-fd7e-445c-8a31-707dc844df12 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:46.129050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a83cac-fd7e-445c-8a31-707dc844df12 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:46.130311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a83cac-fd7e-445c-8a31-707dc844df12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:46.130606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a83cac-fd7e-445c-8a31-707dc844df12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:46.132147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a83cac-fd7e-445c-8a31-707dc844df12 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:46.147710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.sql.core [None req-fd59a346-17ad-451d-a533-ab1d3bd7ba74 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 13:01:46.147710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 13:01:46.147710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [parameters: {'id': '9eede421b8dd4c21b375c8a1fdfd2eca', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 13:01:46.147710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81602) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 13:01:46.151187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 243/476] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:46.153149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a83cac-fd7e-445c-8a31-707dc844df12 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-fd59a346-17ad-451d-a533-ab1d3bd7ba74 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application conn.query(q) Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result.read() Sep 09 13:01:46.170141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 13:01:46.171071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application next(self.gen) Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application yield resource Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application next(self.gen) Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise self.value Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application session.commit() Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 13:01:46.171970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.session.flush() Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application compat.raise_( Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 13:01:46.172803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 13:01:46.173671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application conn.query(q) Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result.read() Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 13:01:46.174492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [parameters: {'id': '9eede421b8dd4c21b375c8a1fdfd2eca', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:46.176816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 13:01:46.178784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:46.181034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 234/477] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 9 13:01:45 2022] POST /identity/v3/roles => generated 144 bytes in 327 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:01:46.181034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59a346-17ad-451d-a533-ab1d3bd7ba74 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:46.189311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:46.273762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a83cac-fd7e-445c-8a31-707dc844df12 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:46.313847 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47727436-b1f2-497c-99e2-047a24673c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:46.315329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727436-b1f2-497c-99e2-047a24673c1d None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:46.315619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727436-b1f2-497c-99e2-047a24673c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:46.315895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47727436-b1f2-497c-99e2-047a24673c1d None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:46.316637 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47727436-b1f2-497c-99e2-047a24673c1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:46.319029 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47727436-b1f2-497c-99e2-047a24673c1d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:46.363915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 244/478] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:01:46 2022] POST /identity/v3/users => generated 249 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:01:46.409762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59a346-17ad-451d-a533-ab1d3bd7ba74 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:46.430494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:46.459709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0838c-7fa4-4603-8491-c5d7926a7d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:46.460378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0838c-7fa4-4603-8491-c5d7926a7d7b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:46.460654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0838c-7fa4-4603-8491-c5d7926a7d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:46.460921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0838c-7fa4-4603-8491-c5d7926a7d7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:46.472359 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0838c-7fa4-4603-8491-c5d7926a7d7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a3a0838c-7fa4-4603-8491-c5d7926a7d7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:46.477336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:46.478243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:46.479333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:46.479333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:46.479333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:46.479333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:46.479333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:46.479333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:46.479333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:46.479333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:46.479333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:46.479333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:46.479333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 13:01:46.479333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:46.479333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 235/479] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:01:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:46.484397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47727436-b1f2-497c-99e2-047a24673c1d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:46.603238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:46.628773 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47727436-b1f2-497c-99e2-047a24673c1d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:46.684685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbdb36-6b80-4bd9-b296-6d128d27d802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:46.685308 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbdb36-6b80-4bd9-b296-6d128d27d802 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:46.685584 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbdb36-6b80-4bd9-b296-6d128d27d802 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:46.685859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbdb36-6b80-4bd9-b296-6d128d27d802 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:46.686536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbdb36-6b80-4bd9-b296-6d128d27d802 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:46.686836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbdb36-6b80-4bd9-b296-6d128d27d802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:46.688234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbdb36-6b80-4bd9-b296-6d128d27d802 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:46.702453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 245/480] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:47.742897 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0d0e2-5e21-4b11-80f3-8963e735ac74 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:47.743433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0d0e2-5e21-4b11-80f3-8963e735ac74 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:47.743507 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0d0e2-5e21-4b11-80f3-8963e735ac74 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:47.745114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 236/481] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:47.761189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9ebd5-e16f-4e22-a80b-e0917239723a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:47.761723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9ebd5-e16f-4e22-a80b-e0917239723a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:47.761723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9ebd5-e16f-4e22-a80b-e0917239723a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:47.788079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ddd9ebd5-e16f-4e22-a80b-e0917239723a None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:47.789312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ddd9ebd5-e16f-4e22-a80b-e0917239723a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:47.882549 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 246/482] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:48.146669 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd5db3-071e-4fb4-a4ce-b93f6a803dee None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:48.147003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd5db3-071e-4fb4-a4ce-b93f6a803dee None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:48.147207 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd5db3-071e-4fb4-a4ce-b93f6a803dee None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:48.148685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 237/483] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:48.153575 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a3df4-e3bc-4afa-8106-26a4084c7489 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:48.153930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a3df4-e3bc-4afa-8106-26a4084c7489 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:48.154211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a3df4-e3bc-4afa-8106-26a4084c7489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:48.180010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-532a3df4-e3bc-4afa-8106-26a4084c7489 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:48.181218 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-532a3df4-e3bc-4afa-8106-26a4084c7489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:48.238559 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236dbf1-3552-4856-9fb9-bd5c451b0cf7 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:48.239457 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236dbf1-3552-4856-9fb9-bd5c451b0cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:48.239791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236dbf1-3552-4856-9fb9-bd5c451b0cf7 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:48.241381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 238/484] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:01:48.252127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a154c-a695-41a4-92f0-6f794deb0671 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:48.252411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a154c-a695-41a4-92f0-6f794deb0671 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:48.252686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a154c-a695-41a4-92f0-6f794deb0671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:48.262758 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 247/485] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:48.288996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a3df4-e3bc-4afa-8106-26a4084c7489 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:48.291388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4d3a154c-a695-41a4-92f0-6f794deb0671 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:48.292553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4d3a154c-a695-41a4-92f0-6f794deb0671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:48.300185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:48.396230 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a3df4-e3bc-4afa-8106-26a4084c7489 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:48.433231 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec3349-59fa-479d-ae93-83ec8d57c1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:48.433869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec3349-59fa-479d-ae93-83ec8d57c1ee None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:48.434143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec3349-59fa-479d-ae93-83ec8d57c1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:48.434454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec3349-59fa-479d-ae93-83ec8d57c1ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:48.437650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 239/486] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:48.446429 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec3349-59fa-479d-ae93-83ec8d57c1ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e6ec3349-59fa-479d-ae93-83ec8d57c1ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:48.451279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:48.452187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:48.453568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:48.453568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:48.453568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:48.453568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:48.453568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:48.453568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:48.453568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:48.453568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:48.453568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:48.453568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:48.453568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:48.453568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:48.453568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 248/487] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:01:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:01:48.458174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a154c-a695-41a4-92f0-6f794deb0671 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:48.574544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:48.581159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a154c-a695-41a4-92f0-6f794deb0671 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:48.608116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec3349-59fa-479d-ae93-83ec8d57c1ee None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:48.621629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd712a28-efb6-4c9d-ad36-748f1ab8e8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:48.622142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd712a28-efb6-4c9d-ad36-748f1ab8e8aa None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:48.622422 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd712a28-efb6-4c9d-ad36-748f1ab8e8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:48.622698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd712a28-efb6-4c9d-ad36-748f1ab8e8aa None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:48.623417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd712a28-efb6-4c9d-ad36-748f1ab8e8aa None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:48.623725 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd712a28-efb6-4c9d-ad36-748f1ab8e8aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:48.625121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd712a28-efb6-4c9d-ad36-748f1ab8e8aa None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:48.637452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 240/488] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:01:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:48.643639 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd712a28-efb6-4c9d-ad36-748f1ab8e8aa None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:48.754563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd712a28-efb6-4c9d-ad36-748f1ab8e8aa None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:48.774647 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:48.793039 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d54a1a-941e-4de4-bae0-12a18c37d33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:48.793677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d54a1a-941e-4de4-bae0-12a18c37d33b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:48.793957 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d54a1a-941e-4de4-bae0-12a18c37d33b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:48.794277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d54a1a-941e-4de4-bae0-12a18c37d33b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:48.805463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d54a1a-941e-4de4-bae0-12a18c37d33b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:48.806988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d54a1a-941e-4de4-bae0-12a18c37d33b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:48.813149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec3349-59fa-479d-ae93-83ec8d57c1ee None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-23d54a1a-941e-4de4-bae0-12a18c37d33b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:48.821961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:48.822846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:48.823721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:48.823721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:48.823721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:48.823721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 241/489] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:48.828850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d54a1a-941e-4de4-bae0-12a18c37d33b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:48.847737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:48.852746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97988883-5ba0-4b47-a578-bd0cab61711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:48.853391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97988883-5ba0-4b47-a578-bd0cab61711f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:48.853672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97988883-5ba0-4b47-a578-bd0cab61711f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:48.853944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97988883-5ba0-4b47-a578-bd0cab61711f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:48.865851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97988883-5ba0-4b47-a578-bd0cab61711f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-97988883-5ba0-4b47-a578-bd0cab61711f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:48.871654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:48.873669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 249/490] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:01:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:48.875622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97988883-5ba0-4b47-a578-bd0cab61711f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:48.958096 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d54a1a-941e-4de4-bae0-12a18c37d33b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:48.990940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97988883-5ba0-4b47-a578-bd0cab61711f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:49.003112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23ff7f-eb00-4ed5-abad-464f98fba74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:49.004202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23ff7f-eb00-4ed5-abad-464f98fba74c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.005302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23ff7f-eb00-4ed5-abad-464f98fba74c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.005633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23ff7f-eb00-4ed5-abad-464f98fba74c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.020480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23ff7f-eb00-4ed5-abad-464f98fba74c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:49.022208 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ec23ff7f-eb00-4ed5-abad-464f98fba74c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:49.029704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:49.030850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:49.031789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:49.031789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:49.031789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:49.031789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:49.031789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:49.031789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:49.031789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:49.031789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:49.031789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:49.031789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:49.031789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:49.031789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:49.032400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384950dd-d519-4daf-8ff1-ade758b022b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:49.032400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384950dd-d519-4daf-8ff1-ade758b022b5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.032400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384950dd-d519-4daf-8ff1-ade758b022b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.032400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384950dd-d519-4daf-8ff1-ade758b022b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.033187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384950dd-d519-4daf-8ff1-ade758b022b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:49.033492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384950dd-d519-4daf-8ff1-ade758b022b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:49.034604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 242/491] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:01:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:49.034870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384950dd-d519-4daf-8ff1-ade758b022b5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:49.035959 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23ff7f-eb00-4ed5-abad-464f98fba74c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:49.048025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 250/492] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:49.048758 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384950dd-d519-4daf-8ff1-ade758b022b5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:49.126686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:49.163361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23ff7f-eb00-4ed5-abad-464f98fba74c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:49.182005 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384950dd-d519-4daf-8ff1-ade758b022b5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:49.210755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f682be-e343-45cf-838e-c75474841e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:49.211985 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f682be-e343-45cf-838e-c75474841e6f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.212867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f682be-e343-45cf-838e-c75474841e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.213655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f682be-e343-45cf-838e-c75474841e6f None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.214828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f682be-e343-45cf-838e-c75474841e6f None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:49.215758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f682be-e343-45cf-838e-c75474841e6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:49.217632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f682be-e343-45cf-838e-c75474841e6f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:49.220712 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46329b98-a2f3-43dc-aebd-9a82bfae2f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:49.221335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46329b98-a2f3-43dc-aebd-9a82bfae2f82 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.221618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46329b98-a2f3-43dc-aebd-9a82bfae2f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.221897 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46329b98-a2f3-43dc-aebd-9a82bfae2f82 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.222571 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46329b98-a2f3-43dc-aebd-9a82bfae2f82 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:49.222878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46329b98-a2f3-43dc-aebd-9a82bfae2f82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:49.224863 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46329b98-a2f3-43dc-aebd-9a82bfae2f82 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:49.229605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 243/493] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:01:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:49.231026 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f682be-e343-45cf-838e-c75474841e6f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:49.235851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 251/494] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:01:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:49.237140 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46329b98-a2f3-43dc-aebd-9a82bfae2f82 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:49.293986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:49.356896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46329b98-a2f3-43dc-aebd-9a82bfae2f82 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:49.374884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f682be-e343-45cf-838e-c75474841e6f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:49.400317 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2362032-52a7-450b-ab3d-043b486500cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:49.401007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2362032-52a7-450b-ab3d-043b486500cb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.401343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2362032-52a7-450b-ab3d-043b486500cb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.401679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2362032-52a7-450b-ab3d-043b486500cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.418346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2362032-52a7-450b-ab3d-043b486500cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:49.420298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2362032-52a7-450b-ab3d-043b486500cb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:49.434043 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f7191-457d-423b-b1ba-fe3f50f20eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:49.434697 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f7191-457d-423b-b1ba-fe3f50f20eca None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.434973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f7191-457d-423b-b1ba-fe3f50f20eca None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.435275 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f7191-457d-423b-b1ba-fe3f50f20eca None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.440049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:49.447064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f7191-457d-423b-b1ba-fe3f50f20eca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e2362032-52a7-450b-ab3d-043b486500cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:49.448496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:49.449371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:49.450252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:49.450252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:49.450252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:49.450415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f7191-457d-423b-b1ba-fe3f50f20eca None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:49.452485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 252/495] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:49.453593 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2362032-52a7-450b-ab3d-043b486500cb None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3e1f7191-457d-423b-b1ba-fe3f50f20eca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:01:49.466885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:49.467865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:49.468762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:49.468762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:49.468762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:01:49.468762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:01:49.468762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:01:49.468762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:01:49.468762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:01:49.468762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:01:49.468762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:01:49.468762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:49.471124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 244/496] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 13:01:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:49.472765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f7191-457d-423b-b1ba-fe3f50f20eca None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:49.577552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2362032-52a7-450b-ab3d-043b486500cb None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:49.578986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:49.590219 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f7191-457d-423b-b1ba-fe3f50f20eca None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:49.618599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf75bb1-7c8e-4a2f-9f87-8aba0c124fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:49.619252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf75bb1-7c8e-4a2f-9f87-8aba0c124fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.619530 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf75bb1-7c8e-4a2f-9f87-8aba0c124fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.619802 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf75bb1-7c8e-4a2f-9f87-8aba0c124fe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.624664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f53ef-3f56-4418-8c24-c35565c9555d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:49.625272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f53ef-3f56-4418-8c24-c35565c9555d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.625552 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f53ef-3f56-4418-8c24-c35565c9555d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.625827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f53ef-3f56-4418-8c24-c35565c9555d None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.626463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f53ef-3f56-4418-8c24-c35565c9555d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:49.626778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f53ef-3f56-4418-8c24-c35565c9555d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:49.628200 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f53ef-3f56-4418-8c24-c35565c9555d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:49.630625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf75bb1-7c8e-4a2f-9f87-8aba0c124fe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:49.632146 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf75bb1-7c8e-4a2f-9f87-8aba0c124fe1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:49.641768 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 245/497] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:49.642660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f53ef-3f56-4418-8c24-c35565c9555d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7bf75bb1-7c8e-4a2f-9f87-8aba0c124fe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:49.647016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:49.647945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:49.648923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:49.648923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:49.648923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:49.652591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 253/498] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:49.653657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4e576-c569-48f1-881b-d5819a47ba1e None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.653933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4e576-c569-48f1-881b-d5819a47ba1e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.654212 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4e576-c569-48f1-881b-d5819a47ba1e None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.655463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 254/499] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:49.656448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fadd11-b99a-42bd-921c-cbd9bf2481c3 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.656731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fadd11-b99a-42bd-921c-cbd9bf2481c3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.656988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fadd11-b99a-42bd-921c-cbd9bf2481c3 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.658145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 255/500] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:49.662568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fadd11-b99a-42bd-921c-cbd9bf2481c3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:49.702033 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:49.745772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f53ef-3f56-4418-8c24-c35565c9555d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:49.764633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fadd11-b99a-42bd-921c-cbd9bf2481c3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:49.780202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe740f4-ae77-439a-9de2-32630f3b9f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:49.781136 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe740f4-ae77-439a-9de2-32630f3b9f9a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.781421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe740f4-ae77-439a-9de2-32630f3b9f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.781692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe740f4-ae77-439a-9de2-32630f3b9f9a None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.782412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe740f4-ae77-439a-9de2-32630f3b9f9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:49.782713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe740f4-ae77-439a-9de2-32630f3b9f9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:49.784178 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe740f4-ae77-439a-9de2-32630f3b9f9a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:49.799338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28eedb-eabe-416e-b3c5-35126e9f201d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:49.799967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28eedb-eabe-416e-b3c5-35126e9f201d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.800244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28eedb-eabe-416e-b3c5-35126e9f201d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.800515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28eedb-eabe-416e-b3c5-35126e9f201d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.808677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28eedb-eabe-416e-b3c5-35126e9f201d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:49.810113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28eedb-eabe-416e-b3c5-35126e9f201d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:49.821092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 246/501] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 13:01:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:49.821919 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe740f4-ae77-439a-9de2-32630f3b9f9a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-fe28eedb-eabe-416e-b3c5-35126e9f201d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:01:49.823436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:49.824312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:49.825277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:49.825277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:49.825277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:01:49.825277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:01:49.825277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:01:49.825277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:01:49.825277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:01:49.825277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:01:49.825277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 13:01:49.825277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:49.825277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 256/502] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:49.825866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4acf9-ab77-49bc-a241-a3bdae3dd185 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.825933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4acf9-ab77-49bc-a241-a3bdae3dd185 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.826194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4acf9-ab77-49bc-a241-a3bdae3dd185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.844135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-63f4acf9-ab77-49bc-a241-a3bdae3dd185 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:49.845257 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-63f4acf9-ab77-49bc-a241-a3bdae3dd185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:49.855510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:49.912685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 257/503] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:49.914358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f50c74-0e44-4d1a-aeff-76ab31b1eb6c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.914639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f50c74-0e44-4d1a-aeff-76ab31b1eb6c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.914911 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f50c74-0e44-4d1a-aeff-76ab31b1eb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.925105 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe740f4-ae77-439a-9de2-32630f3b9f9a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:49.932825 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e2f50c74-0e44-4d1a-aeff-76ab31b1eb6c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:49.934021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e2f50c74-0e44-4d1a-aeff-76ab31b1eb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:49.944487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:49.966871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae3820-881f-4d17-a662-0328f5890472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:49.967729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae3820-881f-4d17-a662-0328f5890472 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:49.968015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae3820-881f-4d17-a662-0328f5890472 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:49.968285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae3820-881f-4d17-a662-0328f5890472 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:49.968927 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae3820-881f-4d17-a662-0328f5890472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:49.969330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae3820-881f-4d17-a662-0328f5890472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:49.971063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae3820-881f-4d17-a662-0328f5890472 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:49.985481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 247/504] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:49.986623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae3820-881f-4d17-a662-0328f5890472 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:50.023078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 258/505] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:50.023078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f50c74-0e44-4d1a-aeff-76ab31b1eb6c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:50.103992 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae3820-881f-4d17-a662-0328f5890472 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:50.109493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:50.138559 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f50c74-0e44-4d1a-aeff-76ab31b1eb6c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:50.140151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45df72-b9a1-4b41-8919-d3a190a240a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:50.140861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45df72-b9a1-4b41-8919-d3a190a240a1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:50.141234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45df72-b9a1-4b41-8919-d3a190a240a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:50.141599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45df72-b9a1-4b41-8919-d3a190a240a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:50.184101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101dd50-3ada-4c3f-8645-d19ba89a477f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:50.184953 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101dd50-3ada-4c3f-8645-d19ba89a477f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:50.185375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101dd50-3ada-4c3f-8645-d19ba89a477f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:50.186271 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101dd50-3ada-4c3f-8645-d19ba89a477f None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:50.187370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101dd50-3ada-4c3f-8645-d19ba89a477f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:50.187689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101dd50-3ada-4c3f-8645-d19ba89a477f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:50.187921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:50.189115 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101dd50-3ada-4c3f-8645-d19ba89a477f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:50.214790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45df72-b9a1-4b41-8919-d3a190a240a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:50.216302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45df72-b9a1-4b41-8919-d3a190a240a1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:50.236805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 259/506] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 9 13:01:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:50.236805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101dd50-3ada-4c3f-8645-d19ba89a477f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:50.326595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 248/507] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:50.327588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45df72-b9a1-4b41-8919-d3a190a240a1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:50.342736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101dd50-3ada-4c3f-8645-d19ba89a477f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:50.372698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:50.378370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4579690-f06f-4172-a661-8edb1edd5411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:50.379005 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4579690-f06f-4172-a661-8edb1edd5411 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:50.380079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4579690-f06f-4172-a661-8edb1edd5411 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:50.380409 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4579690-f06f-4172-a661-8edb1edd5411 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:50.387401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4579690-f06f-4172-a661-8edb1edd5411 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:50.388829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4579690-f06f-4172-a661-8edb1edd5411 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d4579690-f06f-4172-a661-8edb1edd5411 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:01:50.399910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:50.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:50.401747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:50.401747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:50.401747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:01:50.401747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:01:50.401747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:01:50.401747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:01:50.401747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:01:50.401747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:01:50.401747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 13:01:50.401747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:50.402253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:50.402324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 260/508] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:50.402397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4579690-f06f-4172-a661-8edb1edd5411 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:50.427074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45df72-b9a1-4b41-8919-d3a190a240a1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:50.462604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d01ecc-eb6f-4500-9e15-e0b8decbc2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:50.463261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d01ecc-eb6f-4500-9e15-e0b8decbc2fa None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:50.463539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d01ecc-eb6f-4500-9e15-e0b8decbc2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:50.463807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d01ecc-eb6f-4500-9e15-e0b8decbc2fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:50.473705 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d01ecc-eb6f-4500-9e15-e0b8decbc2fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:50.475186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d01ecc-eb6f-4500-9e15-e0b8decbc2fa None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-52d01ecc-eb6f-4500-9e15-e0b8decbc2fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:50.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:50.492493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:50.493374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:50.493374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:50.493374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:50.493374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 249/509] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:50.493374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d01ecc-eb6f-4500-9e15-e0b8decbc2fa None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:50.503247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4579690-f06f-4172-a661-8edb1edd5411 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:50.537020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27002ce-1b46-427a-bca2-1628e543476a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:50.537629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27002ce-1b46-427a-bca2-1628e543476a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:50.537907 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27002ce-1b46-427a-bca2-1628e543476a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:50.538180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27002ce-1b46-427a-bca2-1628e543476a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:50.595999 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d01ecc-eb6f-4500-9e15-e0b8decbc2fa None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:50.597297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:50.610503 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27002ce-1b46-427a-bca2-1628e543476a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:50.612022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27002ce-1b46-427a-bca2-1628e543476a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:50.622424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:50.631678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51bf5a-06ac-4487-9470-1351fd503253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:50.632248 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51bf5a-06ac-4487-9470-1351fd503253 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:50.632526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51bf5a-06ac-4487-9470-1351fd503253 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:50.632797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51bf5a-06ac-4487-9470-1351fd503253 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:50.642810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51bf5a-06ac-4487-9470-1351fd503253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9a51bf5a-06ac-4487-9470-1351fd503253 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:50.648445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:50.649518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 250/510] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:01:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:50.650361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51bf5a-06ac-4487-9470-1351fd503253 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:50.712939 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 261/511] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:50.713860 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27002ce-1b46-427a-bca2-1628e543476a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:50.753099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51bf5a-06ac-4487-9470-1351fd503253 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:50.787798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf5296-f013-488a-b201-a982f6d0898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:50.788406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf5296-f013-488a-b201-a982f6d0898c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:50.788683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf5296-f013-488a-b201-a982f6d0898c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:50.788956 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf5296-f013-488a-b201-a982f6d0898c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:50.799884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf5296-f013-488a-b201-a982f6d0898c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:50.801388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf5296-f013-488a-b201-a982f6d0898c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:50.805327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:50.814144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27002ce-1b46-427a-bca2-1628e543476a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4faf5296-f013-488a-b201-a982f6d0898c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:50.815918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:50.817811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:50.818892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:50.818892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:50.818892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:50.818892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 251/512] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:50.818892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf5296-f013-488a-b201-a982f6d0898c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:50.852774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee9e51-9312-418f-9057-8c12ba7de64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:50.853377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee9e51-9312-418f-9057-8c12ba7de64d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:50.853649 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee9e51-9312-418f-9057-8c12ba7de64d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:50.853914 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee9e51-9312-418f-9057-8c12ba7de64d None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:50.854635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee9e51-9312-418f-9057-8c12ba7de64d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:50.854933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee9e51-9312-418f-9057-8c12ba7de64d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:50.856384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee9e51-9312-418f-9057-8c12ba7de64d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:50.860539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:50.872826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 262/513] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 9 13:01:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:50.873624 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee9e51-9312-418f-9057-8c12ba7de64d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:50.920405 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf5296-f013-488a-b201-a982f6d0898c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:50.956288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc3f0b-9f7c-4e95-a33c-5a24885e4bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:50.956893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc3f0b-9f7c-4e95-a33c-5a24885e4bfe None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:50.957700 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc3f0b-9f7c-4e95-a33c-5a24885e4bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:50.957973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc3f0b-9f7c-4e95-a33c-5a24885e4bfe None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:50.958614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc3f0b-9f7c-4e95-a33c-5a24885e4bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:50.958918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc3f0b-9f7c-4e95-a33c-5a24885e4bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:50.960347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc3f0b-9f7c-4e95-a33c-5a24885e4bfe None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:50.971791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 252/514] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:50.972586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc3f0b-9f7c-4e95-a33c-5a24885e4bfe None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:50.979383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee9e51-9312-418f-9057-8c12ba7de64d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:51.013249 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d9bf1-d2f9-449d-8e1c-8d8286d52336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:51.013852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d9bf1-d2f9-449d-8e1c-8d8286d52336 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:51.014128 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d9bf1-d2f9-449d-8e1c-8d8286d52336 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:51.014399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d9bf1-d2f9-449d-8e1c-8d8286d52336 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:51.015076 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d9bf1-d2f9-449d-8e1c-8d8286d52336 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:51.015382 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d9bf1-d2f9-449d-8e1c-8d8286d52336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:51.016741 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d9bf1-d2f9-449d-8e1c-8d8286d52336 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:51.024914 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 263/515] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:01:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:51.025689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d9bf1-d2f9-449d-8e1c-8d8286d52336 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:51.040556 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:51.070270 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:51.074649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc3f0b-9f7c-4e95-a33c-5a24885e4bfe None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:51.109348 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bb9d7-557d-418b-9461-b896602df00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:51.109980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bb9d7-557d-418b-9461-b896602df00e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:51.110262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bb9d7-557d-418b-9461-b896602df00e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:51.110530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bb9d7-557d-418b-9461-b896602df00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:51.120800 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bb9d7-557d-418b-9461-b896602df00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:51.122267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bb9d7-557d-418b-9461-b896602df00e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:51.125037 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d9bf1-d2f9-449d-8e1c-8d8286d52336 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-455bb9d7-557d-418b-9461-b896602df00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:51.137627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:51.138531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:51.139441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:51.139441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:51.139441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:51.143232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 253/516] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:51.144263 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bb9d7-557d-418b-9461-b896602df00e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:51.159291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0246b3-38f8-4577-87c7-9bf33f6110ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:51.159899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0246b3-38f8-4577-87c7-9bf33f6110ac None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:51.160308 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0246b3-38f8-4577-87c7-9bf33f6110ac None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:51.160657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0246b3-38f8-4577-87c7-9bf33f6110ac None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:51.161355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0246b3-38f8-4577-87c7-9bf33f6110ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:51.161659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0246b3-38f8-4577-87c7-9bf33f6110ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:51.163018 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0246b3-38f8-4577-87c7-9bf33f6110ac None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:51.179328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 264/517] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:51.179328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0246b3-38f8-4577-87c7-9bf33f6110ac None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:51.244813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bb9d7-557d-418b-9461-b896602df00e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:51.245319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:51.279967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0246b3-38f8-4577-87c7-9bf33f6110ac None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:51.280324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23368cf8-0dde-4ee2-8a92-553bb45bacde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:51.280936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23368cf8-0dde-4ee2-8a92-553bb45bacde None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:51.281217 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23368cf8-0dde-4ee2-8a92-553bb45bacde None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:51.281491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23368cf8-0dde-4ee2-8a92-553bb45bacde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:51.301422 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:51.317359 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c0d80-b63e-47e0-bd56-3a533d98ab04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:51.318038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0d80-b63e-47e0-bd56-3a533d98ab04 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:51.318327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0d80-b63e-47e0-bd56-3a533d98ab04 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:51.318602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0d80-b63e-47e0-bd56-3a533d98ab04 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:51.319284 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c0d80-b63e-47e0-bd56-3a533d98ab04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:51.320723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c0d80-b63e-47e0-bd56-3a533d98ab04 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:51.355166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 265/518] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:01:51 2022] POST /identity/v3/users => generated 247 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:01:51.355166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c0d80-b63e-47e0-bd56-3a533d98ab04 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:51.368588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23368cf8-0dde-4ee2-8a92-553bb45bacde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:51.370808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23368cf8-0dde-4ee2-8a92-553bb45bacde None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:51.455690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c0d80-b63e-47e0-bd56-3a533d98ab04 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:51.487901 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:51.489414 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 254/519] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:51.490508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23368cf8-0dde-4ee2-8a92-553bb45bacde None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:51.495935 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18706e05-e978-4a77-89ce-a9173b429abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:51.496587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18706e05-e978-4a77-89ce-a9173b429abe None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:51.496871 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18706e05-e978-4a77-89ce-a9173b429abe None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:51.497139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18706e05-e978-4a77-89ce-a9173b429abe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:51.509244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18706e05-e978-4a77-89ce-a9173b429abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:51.510724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18706e05-e978-4a77-89ce-a9173b429abe None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-18706e05-e978-4a77-89ce-a9173b429abe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:51.526981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:51.527942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:51.528784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:51.528784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:51.528784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:51.528784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 266/520] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:51.528993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18706e05-e978-4a77-89ce-a9173b429abe None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:51.551356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:51.612439 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23368cf8-0dde-4ee2-8a92-553bb45bacde None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:51.648175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b6814-75c6-482f-90bf-538c67505596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:51.648783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b6814-75c6-482f-90bf-538c67505596 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:51.649059 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b6814-75c6-482f-90bf-538c67505596 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:51.649327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b6814-75c6-482f-90bf-538c67505596 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:51.649967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b6814-75c6-482f-90bf-538c67505596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:51.650266 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b6814-75c6-482f-90bf-538c67505596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:51.651680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b6814-75c6-482f-90bf-538c67505596 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:51.652539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18706e05-e978-4a77-89ce-a9173b429abe None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:51.667147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 255/521] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:51.667147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b6814-75c6-482f-90bf-538c67505596 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:51.693300 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb7289-0976-4298-b815-12b301fee541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:51.693933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb7289-0976-4298-b815-12b301fee541 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:51.694217 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb7289-0976-4298-b815-12b301fee541 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:51.694496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb7289-0976-4298-b815-12b301fee541 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:51.701415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eb7289-0976-4298-b815-12b301fee541 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:51.702880 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eb7289-0976-4298-b815-12b301fee541 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-85eb7289-0976-4298-b815-12b301fee541 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:01:51.716619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:51.717477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:51.718337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:01:51.718337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:01:51.718337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:01:51.718337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:01:51.718337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:01:51.718337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:01:51.718337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:51.718337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:51.718337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 267/522] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:51.718808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb7289-0976-4298-b815-12b301fee541 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:51.763474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:51.778490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b6814-75c6-482f-90bf-538c67505596 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:51.789483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:51.814572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d73c2-84ad-4889-aa30-8461b4c03665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:51.815237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d73c2-84ad-4889-aa30-8461b4c03665 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:51.815513 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d73c2-84ad-4889-aa30-8461b4c03665 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:51.815779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d73c2-84ad-4889-aa30-8461b4c03665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:51.827579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d73c2-84ad-4889-aa30-8461b4c03665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:51.828665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb7289-0976-4298-b815-12b301fee541 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:51.829113 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d73c2-84ad-4889-aa30-8461b4c03665 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ec1d73c2-84ad-4889-aa30-8461b4c03665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:51.843827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:51.844766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:51.845601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:51.845601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:51.845601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:51.845601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 256/523] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:51.845826 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d73c2-84ad-4889-aa30-8461b4c03665 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:51.871677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798b397-482d-42d8-8fdc-353c83b050d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:51.872297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798b397-482d-42d8-8fdc-353c83b050d1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:51.872583 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798b397-482d-42d8-8fdc-353c83b050d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:51.872858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798b397-482d-42d8-8fdc-353c83b050d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:51.873509 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798b397-482d-42d8-8fdc-353c83b050d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:51.873818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798b397-482d-42d8-8fdc-353c83b050d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:51.875214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798b397-482d-42d8-8fdc-353c83b050d1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:51.892096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 268/524] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:51.892096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798b397-482d-42d8-8fdc-353c83b050d1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:51.957986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d73c2-84ad-4889-aa30-8461b4c03665 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:51.993527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:51.994897 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84929c-b149-48bd-8755-4095c475d861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:51.995529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84929c-b149-48bd-8755-4095c475d861 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:51.995830 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84929c-b149-48bd-8755-4095c475d861 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:51.996084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84929c-b149-48bd-8755-4095c475d861 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:51.996778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798b397-482d-42d8-8fdc-353c83b050d1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:52.002960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84929c-b149-48bd-8755-4095c475d861 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:52.005076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84929c-b149-48bd-8755-4095c475d861 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ae84929c-b149-48bd-8755-4095c475d861 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:01:52.017123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:52.018018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:52.019621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:01:52.019621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:01:52.019621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:01:52.019621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:01:52.019621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:01:52.019621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:01:52.019621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:52.019621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:52.019621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 257/525] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:52.019621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84929c-b149-48bd-8755-4095c475d861 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:52.038131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17018bf7-3eca-42ea-a7ea-22fcab00c346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:52.038744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17018bf7-3eca-42ea-a7ea-22fcab00c346 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:52.039026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17018bf7-3eca-42ea-a7ea-22fcab00c346 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:52.039325 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17018bf7-3eca-42ea-a7ea-22fcab00c346 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:52.040022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17018bf7-3eca-42ea-a7ea-22fcab00c346 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:52.040329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17018bf7-3eca-42ea-a7ea-22fcab00c346 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:52.041762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17018bf7-3eca-42ea-a7ea-22fcab00c346 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:52.045907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:52.061344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 269/526] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:01:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:52.062489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17018bf7-3eca-42ea-a7ea-22fcab00c346 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:52.129346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84929c-b149-48bd-8755-4095c475d861 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:52.166913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68753bf8-3ddf-4819-b0d3-e285591c74d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:52.167579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68753bf8-3ddf-4819-b0d3-e285591c74d7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:52.167859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68753bf8-3ddf-4819-b0d3-e285591c74d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:52.168154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68753bf8-3ddf-4819-b0d3-e285591c74d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:52.168806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68753bf8-3ddf-4819-b0d3-e285591c74d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:52.169117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68753bf8-3ddf-4819-b0d3-e285591c74d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:52.170509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68753bf8-3ddf-4819-b0d3-e285591c74d7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:52.178581 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17018bf7-3eca-42ea-a7ea-22fcab00c346 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:52.183425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 258/527] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:52.184252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68753bf8-3ddf-4819-b0d3-e285591c74d7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:52.216170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8679e0-b473-4c8b-a46f-0f0f38a303bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:52.216808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8679e0-b473-4c8b-a46f-0f0f38a303bf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:52.217087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8679e0-b473-4c8b-a46f-0f0f38a303bf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:52.217361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8679e0-b473-4c8b-a46f-0f0f38a303bf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:52.225028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8679e0-b473-4c8b-a46f-0f0f38a303bf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:52.226500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8679e0-b473-4c8b-a46f-0f0f38a303bf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-bb8679e0-b473-4c8b-a46f-0f0f38a303bf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:01:52.237890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:52.239436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:52.240368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:52.240368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:52.240368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:01:52.240368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:01:52.240368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:01:52.240368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:01:52.240368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:01:52.240368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:01:52.240368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 13:01:52.240368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:52.240368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 270/528] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:01:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:52.241818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8679e0-b473-4c8b-a46f-0f0f38a303bf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:52.271978 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:52.282206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:52.302750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68753bf8-3ddf-4819-b0d3-e285591c74d7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:52.340559 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c754872-d90b-4d20-84fa-f58006c475df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:52.341423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c754872-d90b-4d20-84fa-f58006c475df None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:52.341699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c754872-d90b-4d20-84fa-f58006c475df None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:52.342024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c754872-d90b-4d20-84fa-f58006c475df None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:52.342720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c754872-d90b-4d20-84fa-f58006c475df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:52.343026 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c754872-d90b-4d20-84fa-f58006c475df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:52.344480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c754872-d90b-4d20-84fa-f58006c475df None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:52.360377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8679e0-b473-4c8b-a46f-0f0f38a303bf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:52.362156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 259/529] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:01:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:52.362858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c754872-d90b-4d20-84fa-f58006c475df None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:52.402569 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af06652-4c60-47c7-9ec8-e005345238f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:52.403199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af06652-4c60-47c7-9ec8-e005345238f4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:52.403477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af06652-4c60-47c7-9ec8-e005345238f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:52.403745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af06652-4c60-47c7-9ec8-e005345238f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:52.404469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af06652-4c60-47c7-9ec8-e005345238f4 None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=441575b8b496457b905e9d5f5846e092) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:52.404830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af06652-4c60-47c7-9ec8-e005345238f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:52.406329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af06652-4c60-47c7-9ec8-e005345238f4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:52.423093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 271/530] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:52 2022] GET /identity/v3/role_assignments?user.id=441575b8b496457b905e9d5f5846e092&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:52.423093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af06652-4c60-47c7-9ec8-e005345238f4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:52.492821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c754872-d90b-4d20-84fa-f58006c475df None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:52.533606 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d6630-3a20-48e4-80a3-c63712ef2b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:52.534184 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d6630-3a20-48e4-80a3-c63712ef2b73 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:52.534468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d6630-3a20-48e4-80a3-c63712ef2b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:52.534856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d6630-3a20-48e4-80a3-c63712ef2b73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:52.542257 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:52.544609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d6630-3a20-48e4-80a3-c63712ef2b73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:52.546636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d6630-3a20-48e4-80a3-c63712ef2b73 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1e1d6630-3a20-48e4-80a3-c63712ef2b73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:01:52.564925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:52.566050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:52.567719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:01:52.567719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:01:52.567719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:01:52.567719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:01:52.567719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:01:52.567719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:01:52.567719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:52.567719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:52.569644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af06652-4c60-47c7-9ec8-e005345238f4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:52.570441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 260/531] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:52.571605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d6630-3a20-48e4-80a3-c63712ef2b73 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:52.575008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:52.617256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7eead-4711-4b31-9faf-9beb433cd6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:52.618000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7eead-4711-4b31-9faf-9beb433cd6fa None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:52.618396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7eead-4711-4b31-9faf-9beb433cd6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:52.618764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7eead-4711-4b31-9faf-9beb433cd6fa None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:52.619641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7eead-4711-4b31-9faf-9beb433cd6fa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:52.620075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7eead-4711-4b31-9faf-9beb433cd6fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:52.621698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7eead-4711-4b31-9faf-9beb433cd6fa None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:52.644858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 272/532] 10.222.0.17 () {56 vars in 1139 bytes} [Fri Sep 9 13:01:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:52.645837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7eead-4711-4b31-9faf-9beb433cd6fa None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:52.712671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d6630-3a20-48e4-80a3-c63712ef2b73 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:52.762408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fa8f7-66ce-4f9a-8da4-9e3c222ba70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:52.763555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fa8f7-66ce-4f9a-8da4-9e3c222ba70b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:52.764002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fa8f7-66ce-4f9a-8da4-9e3c222ba70b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:52.764815 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fa8f7-66ce-4f9a-8da4-9e3c222ba70b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:52.765980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fa8f7-66ce-4f9a-8da4-9e3c222ba70b None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=ea77fa8f5b754a2287f0639717f6e57a) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:52.766443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fa8f7-66ce-4f9a-8da4-9e3c222ba70b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:52.768990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fa8f7-66ce-4f9a-8da4-9e3c222ba70b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:52.770645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7eead-4711-4b31-9faf-9beb433cd6fa None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:52.776942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 261/533] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:52 2022] GET /identity/v3/role_assignments?user.id=ea77fa8f5b754a2287f0639717f6e57a&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:52.778016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fa8f7-66ce-4f9a-8da4-9e3c222ba70b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:52.838492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e6168-a84d-42da-8d06-8d5e4505609d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:52.839163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e6168-a84d-42da-8d06-8d5e4505609d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:52.839357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e6168-a84d-42da-8d06-8d5e4505609d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:52.840557 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e6168-a84d-42da-8d06-8d5e4505609d None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:52.841288 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e6168-a84d-42da-8d06-8d5e4505609d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:52.841602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e6168-a84d-42da-8d06-8d5e4505609d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:52.843117 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e6168-a84d-42da-8d06-8d5e4505609d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:52.872485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 273/534] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:01:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:52.879413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e6168-a84d-42da-8d06-8d5e4505609d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:52.885468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:52.927543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:52.985034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fa8f7-66ce-4f9a-8da4-9e3c222ba70b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:53.024142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aeab49-ce20-481b-9ba8-e78e12938359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:53.024805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aeab49-ce20-481b-9ba8-e78e12938359 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:53.025087 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aeab49-ce20-481b-9ba8-e78e12938359 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:53.025365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aeab49-ce20-481b-9ba8-e78e12938359 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:53.101049 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e6168-a84d-42da-8d06-8d5e4505609d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:53.136318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aeab49-ce20-481b-9ba8-e78e12938359 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:53.137788 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aeab49-ce20-481b-9ba8-e78e12938359 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:53.178522 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814ea95-cfa1-49dc-bb9e-a7f406d612b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:53.179164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814ea95-cfa1-49dc-bb9e-a7f406d612b8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:53.179446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814ea95-cfa1-49dc-bb9e-a7f406d612b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:53.179719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814ea95-cfa1-49dc-bb9e-a7f406d612b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:53.180460 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814ea95-cfa1-49dc-bb9e-a7f406d612b8 None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=a7ef6cd64182474787134e3227beb3aa) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:53.180770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814ea95-cfa1-49dc-bb9e-a7f406d612b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:53.182225 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814ea95-cfa1-49dc-bb9e-a7f406d612b8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:53.192089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 274/535] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:52 2022] GET /identity/v3/role_assignments?user.id=a7ef6cd64182474787134e3227beb3aa&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:53.221892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:53.276707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 262/536] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:53.282831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814ea95-cfa1-49dc-bb9e-a7f406d612b8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:53.371583 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:53.428202 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814ea95-cfa1-49dc-bb9e-a7f406d612b8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:53.466314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e8ba0-5dbf-4acc-b8cd-5f7360493968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:53.466960 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e8ba0-5dbf-4acc-b8cd-5f7360493968 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:53.467269 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e8ba0-5dbf-4acc-b8cd-5f7360493968 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:53.467545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e8ba0-5dbf-4acc-b8cd-5f7360493968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:53.478978 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e8ba0-5dbf-4acc-b8cd-5f7360493968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:53.480511 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e8ba0-5dbf-4acc-b8cd-5f7360493968 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-c04e8ba0-5dbf-4acc-b8cd-5f7360493968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:53.496232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:53.497226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:53.498170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:53.498170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:53.498170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:53.498170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 275/537] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:53.505414 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aeab49-ce20-481b-9ba8-e78e12938359 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:53.620739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aeab49-ce20-481b-9ba8-e78e12938359 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:53.657607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becda77-3141-4690-98c8-7f5d00f5ec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:53.658142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becda77-3141-4690-98c8-7f5d00f5ec32 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:53.658421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becda77-3141-4690-98c8-7f5d00f5ec32 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:53.658696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becda77-3141-4690-98c8-7f5d00f5ec32 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:53.659374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becda77-3141-4690-98c8-7f5d00f5ec32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:53.659681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becda77-3141-4690-98c8-7f5d00f5ec32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:53.661659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becda77-3141-4690-98c8-7f5d00f5ec32 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:53.672595 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c75433-8449-4738-b282-cd4f9a8639f5 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:53.672893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c75433-8449-4738-b282-cd4f9a8639f5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:53.673136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c75433-8449-4738-b282-cd4f9a8639f5 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:53.674099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 263/538] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:53.674651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 276/539] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:01:53.680437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c75433-8449-4738-b282-cd4f9a8639f5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:53.680698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504324f-33d8-4e07-800c-d6f41200f6c0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:53.680987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504324f-33d8-4e07-800c-d6f41200f6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:53.681265 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504324f-33d8-4e07-800c-d6f41200f6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:53.701067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c504324f-33d8-4e07-800c-d6f41200f6c0 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:53.702280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c504324f-33d8-4e07-800c-d6f41200f6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:53.724662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:53.774283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 264/540] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:53.822540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c75433-8449-4738-b282-cd4f9a8639f5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:53.855598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:53.859784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf672c-ecc2-4eac-9691-3a26092997ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:53.860666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf672c-ecc2-4eac-9691-3a26092997ed None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:53.861448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf672c-ecc2-4eac-9691-3a26092997ed None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:53.861933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf672c-ecc2-4eac-9691-3a26092997ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:53.870311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf672c-ecc2-4eac-9691-3a26092997ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:53.872273 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf672c-ecc2-4eac-9691-3a26092997ed None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-4edf672c-ecc2-4eac-9691-3a26092997ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:01:53.887110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:53.888033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:53.888881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:01:53.888881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:01:53.888881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:01:53.888881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:01:53.888881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:01:53.888881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:01:53.888881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:53.888881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:53.891465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 277/541] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:53.898166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504324f-33d8-4e07-800c-d6f41200f6c0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:54.004698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504324f-33d8-4e07-800c-d6f41200f6c0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:54.039503 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5812bf0-10a6-47db-be47-89e1e81a7994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:54.040116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5812bf0-10a6-47db-be47-89e1e81a7994 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:54.040392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5812bf0-10a6-47db-be47-89e1e81a7994 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:54.040660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5812bf0-10a6-47db-be47-89e1e81a7994 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:54.041355 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5812bf0-10a6-47db-be47-89e1e81a7994 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:54.041659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5812bf0-10a6-47db-be47-89e1e81a7994 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:54.043202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5812bf0-10a6-47db-be47-89e1e81a7994 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:54.055888 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 265/542] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:01:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:54.061771 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf672c-ecc2-4eac-9691-3a26092997ed None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:54.165143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf672c-ecc2-4eac-9691-3a26092997ed None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:54.200639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcdae0-ee19-417a-ac33-00b688037147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:54.201249 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcdae0-ee19-417a-ac33-00b688037147 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:54.201528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcdae0-ee19-417a-ac33-00b688037147 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:54.201797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcdae0-ee19-417a-ac33-00b688037147 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:54.202523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcdae0-ee19-417a-ac33-00b688037147 None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=e12deabe077b4095bfa7e2e8fac61d14) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:54.202831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcdae0-ee19-417a-ac33-00b688037147 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:54.204355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcdae0-ee19-417a-ac33-00b688037147 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:54.210848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 278/543] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:54 2022] GET /identity/v3/role_assignments?user.id=e12deabe077b4095bfa7e2e8fac61d14&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:54.281190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5812bf0-10a6-47db-be47-89e1e81a7994 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:54.305784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:54.403457 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5812bf0-10a6-47db-be47-89e1e81a7994 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:54.440168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae35ac8-fa0c-4f7a-b456-5ec58fbae8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:54.440813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae35ac8-fa0c-4f7a-b456-5ec58fbae8eb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:54.441823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae35ac8-fa0c-4f7a-b456-5ec58fbae8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:54.442106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae35ac8-fa0c-4f7a-b456-5ec58fbae8eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:54.454102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae35ac8-fa0c-4f7a-b456-5ec58fbae8eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-eae35ac8-fa0c-4f7a-b456-5ec58fbae8eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:54.460194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:54.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:54.463357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:54.463357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:54.463357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:54.463357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:54.463357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:54.463357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:54.463357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:54.463357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:54.463357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:54.463357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:54.463357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:54.463357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:54.463357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 266/544] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:01:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:54.467164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcdae0-ee19-417a-ac33-00b688037147 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:54.539910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:54.619298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcdae0-ee19-417a-ac33-00b688037147 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:54.652993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81d1ed-e3b2-40c7-8105-963557e63458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:54.653599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81d1ed-e3b2-40c7-8105-963557e63458 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:54.653875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81d1ed-e3b2-40c7-8105-963557e63458 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:54.654143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81d1ed-e3b2-40c7-8105-963557e63458 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:54.654802 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81d1ed-e3b2-40c7-8105-963557e63458 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:54.655123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81d1ed-e3b2-40c7-8105-963557e63458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:54.656488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81d1ed-e3b2-40c7-8105-963557e63458 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:54.665631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 279/545] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:01:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:54.671320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae35ac8-fa0c-4f7a-b456-5ec58fbae8eb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:54.785856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:54.815766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae35ac8-fa0c-4f7a-b456-5ec58fbae8eb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:54.866206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde26a15-f1f2-4b88-af22-05bd14efc551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:54.866841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde26a15-f1f2-4b88-af22-05bd14efc551 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:54.867324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde26a15-f1f2-4b88-af22-05bd14efc551 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:54.867637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde26a15-f1f2-4b88-af22-05bd14efc551 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:54.878771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde26a15-f1f2-4b88-af22-05bd14efc551 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:54.880343 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde26a15-f1f2-4b88-af22-05bd14efc551 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-fde26a15-f1f2-4b88-af22-05bd14efc551 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:54.894968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:54.895898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:54.896736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:54.896736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:54.896736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:54.896736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 267/546] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:54.902028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81d1ed-e3b2-40c7-8105-963557e63458 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:54.968577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75de333-0b30-4a78-a799-3351568630dd None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:54.968884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75de333-0b30-4a78-a799-3351568630dd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:54.969197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75de333-0b30-4a78-a799-3351568630dd None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:54.970632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 268/547] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:54.977120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f5a9d-69d3-4e1a-a434-aec0ed634a64 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:54.977477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f5a9d-69d3-4e1a-a434-aec0ed634a64 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:54.977823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f5a9d-69d3-4e1a-a434-aec0ed634a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:55.009327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81d1ed-e3b2-40c7-8105-963557e63458 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:55.013074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:55.016084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cb7f5a9d-69d3-4e1a-a434-aec0ed634a64 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:55.017304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cb7f5a9d-69d3-4e1a-a434-aec0ed634a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:55.047169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2097da5-d6d7-4366-81ab-662be0b6fa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:55.048385 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2097da5-d6d7-4366-81ab-662be0b6fa55 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:55.048870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2097da5-d6d7-4366-81ab-662be0b6fa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:55.049344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2097da5-d6d7-4366-81ab-662be0b6fa55 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:55.050243 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2097da5-d6d7-4366-81ab-662be0b6fa55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:55.051335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2097da5-d6d7-4366-81ab-662be0b6fa55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:55.052921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2097da5-d6d7-4366-81ab-662be0b6fa55 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:55.077411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 280/548] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:55.085151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2097da5-d6d7-4366-81ab-662be0b6fa55 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:55.130395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 269/549] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:01:55.220827 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2097da5-d6d7-4366-81ab-662be0b6fa55 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:55.257717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83abb0df-06bf-4353-a81b-87dc77739969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:55.258745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abb0df-06bf-4353-a81b-87dc77739969 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:55.259278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abb0df-06bf-4353-a81b-87dc77739969 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:55.259771 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abb0df-06bf-4353-a81b-87dc77739969 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:55.267338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83abb0df-06bf-4353-a81b-87dc77739969 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:55.269188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83abb0df-06bf-4353-a81b-87dc77739969 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-83abb0df-06bf-4353-a81b-87dc77739969 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:01:55.282169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:55.283735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:55.285279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:55.285279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:55.285279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:01:55.285279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:01:55.285279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:01:55.285279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:01:55.285279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:01:55.285279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:01:55.285279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 13:01:55.285279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:55.288019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 281/550] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:01:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:01:55.295393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f5a9d-69d3-4e1a-a434-aec0ed634a64 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:55.392797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:55.454264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f5a9d-69d3-4e1a-a434-aec0ed634a64 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:55.504887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd2b00-5dec-41db-90cb-cb4ea3260ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:55.505503 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd2b00-5dec-41db-90cb-cb4ea3260ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:55.505843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd2b00-5dec-41db-90cb-cb4ea3260ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:55.506115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd2b00-5dec-41db-90cb-cb4ea3260ad0 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:55.506843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd2b00-5dec-41db-90cb-cb4ea3260ad0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:55.507167 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd2b00-5dec-41db-90cb-cb4ea3260ad0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:55.508593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd2b00-5dec-41db-90cb-cb4ea3260ad0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:55.533955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 270/551] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 9 13:01:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:55.539739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83abb0df-06bf-4353-a81b-87dc77739969 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:55.586327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:55.601536 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd2b00-5dec-41db-90cb-cb4ea3260ad0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:55.689676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83abb0df-06bf-4353-a81b-87dc77739969 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:55.712470 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd2b00-5dec-41db-90cb-cb4ea3260ad0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:55.730057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fde0b-3695-4ac5-b9ed-c7f025af5a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:55.731114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fde0b-3695-4ac5-b9ed-c7f025af5a20 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:55.731735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fde0b-3695-4ac5-b9ed-c7f025af5a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:55.732203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fde0b-3695-4ac5-b9ed-c7f025af5a20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:55.734215 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:55.757321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74d513-c892-4645-a816-97404e364b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:55.757960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74d513-c892-4645-a816-97404e364b70 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:55.758253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74d513-c892-4645-a816-97404e364b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:55.758526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74d513-c892-4645-a816-97404e364b70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:55.775628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74d513-c892-4645-a816-97404e364b70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ae74d513-c892-4645-a816-97404e364b70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:55.782702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:55.783759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:55.784666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:55.784666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:55.784666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:55.784666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:55.784666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:55.784666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:55.784666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:55.784666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:55.784666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:55.784666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:55.784666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:55.784666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:55.784666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 271/552] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:01:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:55.785589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6a7c5-6433-444a-9488-a9e0df9d295e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:55.790728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6a7c5-6433-444a-9488-a9e0df9d295e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:55.790728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6a7c5-6433-444a-9488-a9e0df9d295e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:55.791226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 272/553] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:55.792357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6a7c5-6433-444a-9488-a9e0df9d295e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:55.826336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fde0b-3695-4ac5-b9ed-c7f025af5a20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:55.829175 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fde0b-3695-4ac5-b9ed-c7f025af5a20 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:55.855830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:55.920004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6a7c5-6433-444a-9488-a9e0df9d295e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:55.960540 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36050c-f896-4bb5-8fba-69272a1d935b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:55.961514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36050c-f896-4bb5-8fba-69272a1d935b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:55.961996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36050c-f896-4bb5-8fba-69272a1d935b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:55.962467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36050c-f896-4bb5-8fba-69272a1d935b None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:55.963364 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36050c-f896-4bb5-8fba-69272a1d935b None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:55.963866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36050c-f896-4bb5-8fba-69272a1d935b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:55.965219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 282/554] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:01:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:01:55.966331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36050c-f896-4bb5-8fba-69272a1d935b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:55.966454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760dfa9-c0c5-4e50-b30c-094c6c3dc39e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:55.966628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760dfa9-c0c5-4e50-b30c-094c6c3dc39e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:55.966877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760dfa9-c0c5-4e50-b30c-094c6c3dc39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:55.981370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 273/555] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:01:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:55.982636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36050c-f896-4bb5-8fba-69272a1d935b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:55.990349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4760dfa9-c0c5-4e50-b30c-094c6c3dc39e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:55.991628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4760dfa9-c0c5-4e50-b30c-094c6c3dc39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:56.027334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:56.090458 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 283/556] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:56.090458 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760dfa9-c0c5-4e50-b30c-094c6c3dc39e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:56.122484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36050c-f896-4bb5-8fba-69272a1d935b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:56.165911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba7eae-97be-4064-a895-89495e6941b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:56.166660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba7eae-97be-4064-a895-89495e6941b2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:56.166947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba7eae-97be-4064-a895-89495e6941b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:56.167254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba7eae-97be-4064-a895-89495e6941b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:56.175989 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:56.181212 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba7eae-97be-4064-a895-89495e6941b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:56.183684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba7eae-97be-4064-a895-89495e6941b2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-54ba7eae-97be-4064-a895-89495e6941b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:56.202623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:56.203813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:56.204730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:56.204730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:56.204730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:56.207251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 274/557] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:56.208939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408cbb8-f13c-4ab5-855e-48fced2f6a1f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:56.209220 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408cbb8-f13c-4ab5-855e-48fced2f6a1f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:56.209494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408cbb8-f13c-4ab5-855e-48fced2f6a1f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:56.210810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 275/558] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:56.211696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408cbb8-f13c-4ab5-855e-48fced2f6a1f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:56.239767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760dfa9-c0c5-4e50-b30c-094c6c3dc39e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:56.278117 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd02107-a7b8-4e27-b3e9-4bcfd391af39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:56.278940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd02107-a7b8-4e27-b3e9-4bcfd391af39 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:56.279264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd02107-a7b8-4e27-b3e9-4bcfd391af39 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:56.279539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd02107-a7b8-4e27-b3e9-4bcfd391af39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:56.291537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd02107-a7b8-4e27-b3e9-4bcfd391af39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:56.292998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd02107-a7b8-4e27-b3e9-4bcfd391af39 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0dd02107-a7b8-4e27-b3e9-4bcfd391af39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:56.309893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:56.310925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:56.311853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:56.311853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:56.311853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:56.311853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 284/559] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:56.313104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6fbc9-0bee-4285-9702-1476d8ffe180 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:56.313665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6fbc9-0bee-4285-9702-1476d8ffe180 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:56.317048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6fbc9-0bee-4285-9702-1476d8ffe180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:56.322835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:56.337405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0bf6fbc9-0bee-4285-9702-1476d8ffe180 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:56.338677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0bf6fbc9-0bee-4285-9702-1476d8ffe180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:56.341681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408cbb8-f13c-4ab5-855e-48fced2f6a1f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:56.380003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281120c9-cdc8-49ca-8efa-fa1d8d0fa275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:56.381082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281120c9-cdc8-49ca-8efa-fa1d8d0fa275 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:56.381558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281120c9-cdc8-49ca-8efa-fa1d8d0fa275 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:56.382130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281120c9-cdc8-49ca-8efa-fa1d8d0fa275 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:56.382967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281120c9-cdc8-49ca-8efa-fa1d8d0fa275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:56.383493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281120c9-cdc8-49ca-8efa-fa1d8d0fa275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:56.385076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281120c9-cdc8-49ca-8efa-fa1d8d0fa275 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:56.399500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 276/560] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:56.400634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281120c9-cdc8-49ca-8efa-fa1d8d0fa275 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:56.417411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 285/561] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:56.418253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6fbc9-0bee-4285-9702-1476d8ffe180 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:56.463159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:56.533052 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281120c9-cdc8-49ca-8efa-fa1d8d0fa275 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:56.534547 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6fbc9-0bee-4285-9702-1476d8ffe180 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:56.572291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3543ce9-1f75-4a6f-ac48-bff27a80c38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:56.573184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3543ce9-1f75-4a6f-ac48-bff27a80c38a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:56.573616 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3543ce9-1f75-4a6f-ac48-bff27a80c38a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:56.574028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3543ce9-1f75-4a6f-ac48-bff27a80c38a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:56.578391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9199b6-e49a-4802-8f45-b80e54278784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:56.579264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9199b6-e49a-4802-8f45-b80e54278784 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:56.579770 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9199b6-e49a-4802-8f45-b80e54278784 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:56.580137 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9199b6-e49a-4802-8f45-b80e54278784 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:56.580877 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9199b6-e49a-4802-8f45-b80e54278784 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:56.581836 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9199b6-e49a-4802-8f45-b80e54278784 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:56.583988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9199b6-e49a-4802-8f45-b80e54278784 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:56.587611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3543ce9-1f75-4a6f-ac48-bff27a80c38a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e3543ce9-1f75-4a6f-ac48-bff27a80c38a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:56.596563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:56.598451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:56.599318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:56.599318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:56.599318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:56.599318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:56.599318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:56.599318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:56.599318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:56.599318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:56.599318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:56.599318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:56.599318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:56.599318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:56.603288 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 286/562] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:01:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:56.604344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3543ce9-1f75-4a6f-ac48-bff27a80c38a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:56.616774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 277/563] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:56.616774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9199b6-e49a-4802-8f45-b80e54278784 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:56.632557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:56.723759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:56.729825 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3543ce9-1f75-4a6f-ac48-bff27a80c38a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:56.746547 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9199b6-e49a-4802-8f45-b80e54278784 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:56.775222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea361916-cea9-4ac0-bcdf-c6cdccaf3934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:56.775891 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea361916-cea9-4ac0-bcdf-c6cdccaf3934 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:56.776177 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea361916-cea9-4ac0-bcdf-c6cdccaf3934 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:56.776453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea361916-cea9-4ac0-bcdf-c6cdccaf3934 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:56.784838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea361916-cea9-4ac0-bcdf-c6cdccaf3934 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:56.786938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea361916-cea9-4ac0-bcdf-c6cdccaf3934 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:56.789799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc51f25-4171-4567-ae3a-cb5ec6780019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:56.790473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc51f25-4171-4567-ae3a-cb5ec6780019 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:56.790816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc51f25-4171-4567-ae3a-cb5ec6780019 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:56.791178 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc51f25-4171-4567-ae3a-cb5ec6780019 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:56.791941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc51f25-4171-4567-ae3a-cb5ec6780019 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:56.792309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc51f25-4171-4567-ae3a-cb5ec6780019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:56.793737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc51f25-4171-4567-ae3a-cb5ec6780019 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ea361916-cea9-4ac0-bcdf-c6cdccaf3934 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:01:56.798519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:56.799672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:56.801927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:01:56.801927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:01:56.801927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:01:56.801927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:01:56.801927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:01:56.801927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:01:56.801927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:56.801927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:56.801927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 287/564] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:56.804648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea361916-cea9-4ac0-bcdf-c6cdccaf3934 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:56.808694 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 278/565] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:01:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:56.808694 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc51f25-4171-4567-ae3a-cb5ec6780019 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:56.875982 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:56.906836 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc51f25-4171-4567-ae3a-cb5ec6780019 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:56.908977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea361916-cea9-4ac0-bcdf-c6cdccaf3934 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:56.942160 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220c036-5338-407e-9e36-99c80e7ed535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:56.942796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220c036-5338-407e-9e36-99c80e7ed535 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:56.943103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220c036-5338-407e-9e36-99c80e7ed535 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:56.943383 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220c036-5338-407e-9e36-99c80e7ed535 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:56.944529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a1666-51c5-4eb2-bff8-41bbdfdd4d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:56.945154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a1666-51c5-4eb2-bff8-41bbdfdd4d58 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:56.945427 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a1666-51c5-4eb2-bff8-41bbdfdd4d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:56.945693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a1666-51c5-4eb2-bff8-41bbdfdd4d58 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:56.953039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a1666-51c5-4eb2-bff8-41bbdfdd4d58 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:56.953441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220c036-5338-407e-9e36-99c80e7ed535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:56.954458 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a1666-51c5-4eb2-bff8-41bbdfdd4d58 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3220c036-5338-407e-9e36-99c80e7ed535 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:56.959063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:56.959996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:56.960819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:56.960819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:56.960819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:56.960819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:56.960819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:56.960819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:56.960819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:56.960819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:56.960819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:56.960819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:56.960819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:56.960819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:56.960819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 279/566] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:01:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:56.961511 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220c036-5338-407e-9e36-99c80e7ed535 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-4d1a1666-51c5-4eb2-bff8-41bbdfdd4d58 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:01:56.967695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:56.968979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:56.969930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:56.969930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:56.969930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:01:56.969930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:01:56.969930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:01:56.969930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:01:56.969930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:01:56.969930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:01:56.969930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:01:56.969930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:56.969930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 288/567] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 13:01:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:56.969930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a1666-51c5-4eb2-bff8-41bbdfdd4d58 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:56.990431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:57.063233 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220c036-5338-407e-9e36-99c80e7ed535 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:57.073675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a1666-51c5-4eb2-bff8-41bbdfdd4d58 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:57.092642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:57.098049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1eb5f5-45c0-493d-b2d1-b06402ef32b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:57.098653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1eb5f5-45c0-493d-b2d1-b06402ef32b7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.098929 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1eb5f5-45c0-493d-b2d1-b06402ef32b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.099244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1eb5f5-45c0-493d-b2d1-b06402ef32b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:57.099940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1eb5f5-45c0-493d-b2d1-b06402ef32b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:57.100246 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1eb5f5-45c0-493d-b2d1-b06402ef32b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:57.101694 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1eb5f5-45c0-493d-b2d1-b06402ef32b7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:57.106862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36dcd-c3b5-4507-bd02-d47c54b5d8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:57.107528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36dcd-c3b5-4507-bd02-d47c54b5d8cf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.107807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36dcd-c3b5-4507-bd02-d47c54b5d8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.108079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36dcd-c3b5-4507-bd02-d47c54b5d8cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:57.114079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 280/568] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:01:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:57.115377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49b0c2-b72a-42a6-9b4e-d9e008d5ccc6 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.115672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49b0c2-b72a-42a6-9b4e-d9e008d5ccc6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.115948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49b0c2-b72a-42a6-9b4e-d9e008d5ccc6 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:57.117177 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 281/569] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:57.117848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49b0c2-b72a-42a6-9b4e-d9e008d5ccc6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:57.124742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a36dcd-c3b5-4507-bd02-d47c54b5d8cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:57.126205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a36dcd-c3b5-4507-bd02-d47c54b5d8cf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-82a36dcd-c3b5-4507-bd02-d47c54b5d8cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:57.140943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:57.141932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:57.142845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:57.142845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:57.142845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:57.142845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 289/570] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:57.142845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36dcd-c3b5-4507-bd02-d47c54b5d8cf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:57.209907 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:57.225337 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49b0c2-b72a-42a6-9b4e-d9e008d5ccc6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:57.243900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36dcd-c3b5-4507-bd02-d47c54b5d8cf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:57.258822 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24787185-2e0c-4a44-b249-8238d43f51b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:57.259456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24787185-2e0c-4a44-b249-8238d43f51b3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.259730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24787185-2e0c-4a44-b249-8238d43f51b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.259998 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24787185-2e0c-4a44-b249-8238d43f51b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:57.260658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24787185-2e0c-4a44-b249-8238d43f51b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:57.260957 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24787185-2e0c-4a44-b249-8238d43f51b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:57.262321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24787185-2e0c-4a44-b249-8238d43f51b3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:57.270500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 282/571] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:01:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:57.271304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24787185-2e0c-4a44-b249-8238d43f51b3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:57.282054 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320f28c-3fa0-4ea9-8a3d-a21e6f705646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:57.282656 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320f28c-3fa0-4ea9-8a3d-a21e6f705646 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.282937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320f28c-3fa0-4ea9-8a3d-a21e6f705646 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.283241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320f28c-3fa0-4ea9-8a3d-a21e6f705646 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:57.283962 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320f28c-3fa0-4ea9-8a3d-a21e6f705646 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:57.285503 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320f28c-3fa0-4ea9-8a3d-a21e6f705646 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:57.286992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320f28c-3fa0-4ea9-8a3d-a21e6f705646 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:57.311168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 290/572] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 13:01:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:57.312396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bebff-5127-4b19-b7fa-31ad43236bfd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.312677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bebff-5127-4b19-b7fa-31ad43236bfd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.312947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bebff-5127-4b19-b7fa-31ad43236bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:57.326901 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:57.332287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-bd4bebff-5127-4b19-b7fa-31ad43236bfd None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:57.333517 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bd4bebff-5127-4b19-b7fa-31ad43236bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:57.372881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24787185-2e0c-4a44-b249-8238d43f51b3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:57.398932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 291/573] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:01:57.399792 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bebff-5127-4b19-b7fa-31ad43236bfd None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:57.419960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3623f25-ed3f-4002-81a0-7dc1e7c3d079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:57.420561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3623f25-ed3f-4002-81a0-7dc1e7c3d079 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.420839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3623f25-ed3f-4002-81a0-7dc1e7c3d079 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.421109 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3623f25-ed3f-4002-81a0-7dc1e7c3d079 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:57.421832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3623f25-ed3f-4002-81a0-7dc1e7c3d079 None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=c361a3e83f8e45c2b4d28f97f0a9d069) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:57.422139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3623f25-ed3f-4002-81a0-7dc1e7c3d079 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:57.423609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3623f25-ed3f-4002-81a0-7dc1e7c3d079 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:57.429109 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 283/574] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:01:57 2022] GET /identity/v3/role_assignments?user.id=c361a3e83f8e45c2b4d28f97f0a9d069&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:57.429890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3623f25-ed3f-4002-81a0-7dc1e7c3d079 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:57.480164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:57.534518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bebff-5127-4b19-b7fa-31ad43236bfd None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:57.566981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3623f25-ed3f-4002-81a0-7dc1e7c3d079 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:57.578913 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39077f26-538c-4467-8713-98f1e52a60c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:57.579653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39077f26-538c-4467-8713-98f1e52a60c0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.579935 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39077f26-538c-4467-8713-98f1e52a60c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.580204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39077f26-538c-4467-8713-98f1e52a60c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:57.581924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39077f26-538c-4467-8713-98f1e52a60c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:57.581924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39077f26-538c-4467-8713-98f1e52a60c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:57.583223 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39077f26-538c-4467-8713-98f1e52a60c0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:57.592592 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:57.607077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 292/575] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:57.607077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39077f26-538c-4467-8713-98f1e52a60c0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:57.633986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14606b7d-a7ac-46e5-a4d9-28ccc29d1b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:57.634599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14606b7d-a7ac-46e5-a4d9-28ccc29d1b22 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.634877 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14606b7d-a7ac-46e5-a4d9-28ccc29d1b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.635181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14606b7d-a7ac-46e5-a4d9-28ccc29d1b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:57.647866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14606b7d-a7ac-46e5-a4d9-28ccc29d1b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:57.649330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14606b7d-a7ac-46e5-a4d9-28ccc29d1b22 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-14606b7d-a7ac-46e5-a4d9-28ccc29d1b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:57.664162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:57.665120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:57.666042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:57.666042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:57.666042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:57.670078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 284/576] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:57.670956 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14606b7d-a7ac-46e5-a4d9-28ccc29d1b22 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:57.726890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39077f26-538c-4467-8713-98f1e52a60c0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:57.745566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:57.766344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3fe04-b870-47c4-b2a3-99d756f3f645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:57.766976 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3fe04-b870-47c4-b2a3-99d756f3f645 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.767283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3fe04-b870-47c4-b2a3-99d756f3f645 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.767555 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3fe04-b870-47c4-b2a3-99d756f3f645 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:57.774318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14606b7d-a7ac-46e5-a4d9-28ccc29d1b22 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:57.779364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3fe04-b870-47c4-b2a3-99d756f3f645 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:57.780947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3fe04-b870-47c4-b2a3-99d756f3f645 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e8d3fe04-b870-47c4-b2a3-99d756f3f645 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:57.796611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:57.797572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:57.798469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:57.798469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:57.798469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:57.798469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 293/577] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:57.798702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3fe04-b870-47c4-b2a3-99d756f3f645 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:57.814244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d0663-73ad-4034-85ba-c409707dfc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:57.814877 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d0663-73ad-4034-85ba-c409707dfc38 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.815179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d0663-73ad-4034-85ba-c409707dfc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.815478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d0663-73ad-4034-85ba-c409707dfc38 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:57.823822 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d0663-73ad-4034-85ba-c409707dfc38 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:57.825268 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d0663-73ad-4034-85ba-c409707dfc38 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-fa9d0663-73ad-4034-85ba-c409707dfc38 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:01:57.836820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:57.837764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:57.839388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:57.839388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:57.839388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:01:57.839388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:01:57.839388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:01:57.839388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:01:57.839388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:01:57.839388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:01:57.839388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 13:01:57.839388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:57.839388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 285/578] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:57.839388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d0663-73ad-4034-85ba-c409707dfc38 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:57.874418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:57.912485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3fe04-b870-47c4-b2a3-99d756f3f645 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:57.953967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d0663-73ad-4034-85ba-c409707dfc38 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:57.958896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08732e6-b854-41c3-a561-1d181c2d9503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:57.959550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08732e6-b854-41c3-a561-1d181c2d9503 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.959829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08732e6-b854-41c3-a561-1d181c2d9503 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.960100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08732e6-b854-41c3-a561-1d181c2d9503 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:57.960763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08732e6-b854-41c3-a561-1d181c2d9503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:57.961067 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08732e6-b854-41c3-a561-1d181c2d9503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:57.962861 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08732e6-b854-41c3-a561-1d181c2d9503 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:57.981087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 294/579] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:57.981087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08732e6-b854-41c3-a561-1d181c2d9503 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:57.996262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5148864-d062-4230-b6c0-0ad0d74d776f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:57.996899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5148864-d062-4230-b6c0-0ad0d74d776f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.997180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5148864-d062-4230-b6c0-0ad0d74d776f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.997454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5148864-d062-4230-b6c0-0ad0d74d776f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:58.008468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:58.009946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5148864-d062-4230-b6c0-0ad0d74d776f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b5148864-d062-4230-b6c0-0ad0d74d776f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:01:58.015117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:58.016031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:58.016878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:58.016878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:58.016878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:58.016878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:58.016878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:01:58.016878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:01:58.016878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:01:58.016878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:01:58.016878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:01:58.016878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:01:58.016878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:01:58.016878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:58.016878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 286/580] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:01:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:58.017583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5148864-d062-4230-b6c0-0ad0d74d776f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:58.089911 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08732e6-b854-41c3-a561-1d181c2d9503 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:58.119841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:58.127770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4f4e6-f376-49c2-91f0-7b3d6d11b2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:58.128914 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4f4e6-f376-49c2-91f0-7b3d6d11b2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:58.129395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4f4e6-f376-49c2-91f0-7b3d6d11b2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:58.129533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4f4e6-f376-49c2-91f0-7b3d6d11b2f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:58.130203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4f4e6-f376-49c2-91f0-7b3d6d11b2f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:58.130514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4f4e6-f376-49c2-91f0-7b3d6d11b2f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:58.131918 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4f4e6-f376-49c2-91f0-7b3d6d11b2f1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:58.132608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5148864-d062-4230-b6c0-0ad0d74d776f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:58.145382 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 295/581] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:58.146323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4f4e6-f376-49c2-91f0-7b3d6d11b2f1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:58.180308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b5e7-0962-43bf-a088-51a94771bfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:58.181567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b5e7-0962-43bf-a088-51a94771bfbd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:58.182474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b5e7-0962-43bf-a088-51a94771bfbd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:58.183253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b5e7-0962-43bf-a088-51a94771bfbd None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:58.184124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630b5e7-0962-43bf-a088-51a94771bfbd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:58.184576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630b5e7-0962-43bf-a088-51a94771bfbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:58.187312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630b5e7-0962-43bf-a088-51a94771bfbd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:58.209810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 287/582] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 9 13:01:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:58.209810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b5e7-0962-43bf-a088-51a94771bfbd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:58.248348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:58.263841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4f4e6-f376-49c2-91f0-7b3d6d11b2f1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:58.304109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2bb85-fcde-4568-98f9-e74a22f985b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:58.305177 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2bb85-fcde-4568-98f9-e74a22f985b8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:58.305517 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2bb85-fcde-4568-98f9-e74a22f985b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:58.306048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2bb85-fcde-4568-98f9-e74a22f985b8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:58.310486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b5e7-0962-43bf-a088-51a94771bfbd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:58.315435 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f2bb85-fcde-4568-98f9-e74a22f985b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:58.316887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f2bb85-fcde-4568-98f9-e74a22f985b8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-67f2bb85-fcde-4568-98f9-e74a22f985b8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:01:58.330413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:58.331977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:58.333295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:58.333295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:58.333295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:01:58.333295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:01:58.333295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:01:58.333295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:01:58.333295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:01:58.333295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:01:58.333295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 13:01:58.333295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:58.337444 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 296/583] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:01:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:58.338606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2bb85-fcde-4568-98f9-e74a22f985b8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:58.352730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dce315-de25-4b27-a775-d2b1953e7d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:58.353432 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dce315-de25-4b27-a775-d2b1953e7d08 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:58.353811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dce315-de25-4b27-a775-d2b1953e7d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:58.354182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dce315-de25-4b27-a775-d2b1953e7d08 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:58.354928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dce315-de25-4b27-a775-d2b1953e7d08 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:58.355324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dce315-de25-4b27-a775-d2b1953e7d08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:58.356781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dce315-de25-4b27-a775-d2b1953e7d08 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:58.369905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 288/584] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:01:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:58.369905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dce315-de25-4b27-a775-d2b1953e7d08 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:58.380560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:58.455796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2bb85-fcde-4568-98f9-e74a22f985b8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:58.473942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dce315-de25-4b27-a775-d2b1953e7d08 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:58.492119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623143cc-bc77-4fbb-9b03-c90262144aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:58.493007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623143cc-bc77-4fbb-9b03-c90262144aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:58.493434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623143cc-bc77-4fbb-9b03-c90262144aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:58.493857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623143cc-bc77-4fbb-9b03-c90262144aa9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:58.500694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623143cc-bc77-4fbb-9b03-c90262144aa9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:58.502311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623143cc-bc77-4fbb-9b03-c90262144aa9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:58.509801 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d557f2f-17da-48f4-980a-6032f8d8eea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:58.510423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d557f2f-17da-48f4-980a-6032f8d8eea1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:58.510698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d557f2f-17da-48f4-980a-6032f8d8eea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:58.510962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d557f2f-17da-48f4-980a-6032f8d8eea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-623143cc-bc77-4fbb-9b03-c90262144aa9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:01:58.515299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:58.516352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:58.517457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:01:58.517457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:01:58.517457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:01:58.517457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:01:58.517457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:01:58.517457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:01:58.517457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:58.517457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:58.519499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 297/585] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:58.520475 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623143cc-bc77-4fbb-9b03-c90262144aa9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:58.527084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d557f2f-17da-48f4-980a-6032f8d8eea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:58.528516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d557f2f-17da-48f4-980a-6032f8d8eea1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:58.533777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1d557f2f-17da-48f4-980a-6032f8d8eea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:58.543662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:58.544876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:58.545792 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:58.545792 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:58.545792 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:58.545792 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 289/586] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:58.546014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d557f2f-17da-48f4-980a-6032f8d8eea1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:58.616086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:58.642158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623143cc-bc77-4fbb-9b03-c90262144aa9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:58.662391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d557f2f-17da-48f4-980a-6032f8d8eea1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:58.682122 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039b943-0fc2-4416-acac-cfbc8f6ec9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:58.682990 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039b943-0fc2-4416-acac-cfbc8f6ec9ee None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:58.683395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039b943-0fc2-4416-acac-cfbc8f6ec9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:58.683817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039b943-0fc2-4416-acac-cfbc8f6ec9ee None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:58.684650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039b943-0fc2-4416-acac-cfbc8f6ec9ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:58.685057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039b943-0fc2-4416-acac-cfbc8f6ec9ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:58.686835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039b943-0fc2-4416-acac-cfbc8f6ec9ee None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:58.697413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9b1d4-3755-47bc-9365-9a7a3b641e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:58.698152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9b1d4-3755-47bc-9365-9a7a3b641e82 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:58.698508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9b1d4-3755-47bc-9365-9a7a3b641e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:58.698782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9b1d4-3755-47bc-9365-9a7a3b641e82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:58.710634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9b1d4-3755-47bc-9365-9a7a3b641e82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:58.712155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9b1d4-3755-47bc-9365-9a7a3b641e82 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:58.712837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 298/587] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 9 13:01:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:58.713582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039b943-0fc2-4416-acac-cfbc8f6ec9ee None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d0f9b1d4-3755-47bc-9365-9a7a3b641e82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:58.727088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:58.728297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:58.729546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:58.729546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:58.729546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:01:58.729546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 290/588] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:58.729546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9b1d4-3755-47bc-9365-9a7a3b641e82 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:58.789417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:58.826269 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039b943-0fc2-4416-acac-cfbc8f6ec9ee None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:58.837348 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9b1d4-3755-47bc-9365-9a7a3b641e82 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:58.851828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:58.863274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0242e43-e758-41a2-a42b-7d8a226f8e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:58.864075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0242e43-e758-41a2-a42b-7d8a226f8e68 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:58.864363 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0242e43-e758-41a2-a42b-7d8a226f8e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:58.864630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0242e43-e758-41a2-a42b-7d8a226f8e68 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:58.865318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0242e43-e758-41a2-a42b-7d8a226f8e68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:58.865627 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0242e43-e758-41a2-a42b-7d8a226f8e68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:58.867072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0242e43-e758-41a2-a42b-7d8a226f8e68 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:58.872983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6cb431-2cdb-40b8-abdd-47c334a16c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:58.873593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cb431-2cdb-40b8-abdd-47c334a16c5c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:58.873868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cb431-2cdb-40b8-abdd-47c334a16c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:58.874199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cb431-2cdb-40b8-abdd-47c334a16c5c None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:58.874837 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6cb431-2cdb-40b8-abdd-47c334a16c5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:58.875170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6cb431-2cdb-40b8-abdd-47c334a16c5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:58.876524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6cb431-2cdb-40b8-abdd-47c334a16c5c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:58.879668 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 299/589] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:01:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:58.880229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0242e43-e758-41a2-a42b-7d8a226f8e68 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:58.894916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 291/590] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:58.900040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6cb431-2cdb-40b8-abdd-47c334a16c5c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:58.987783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0242e43-e758-41a2-a42b-7d8a226f8e68 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:58.998936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6cb431-2cdb-40b8-abdd-47c334a16c5c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:59.014859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:59.025450 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e73ca7-65c8-44e1-a4d0-dc01610440f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:59.026465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e73ca7-65c8-44e1-a4d0-dc01610440f2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:59.026815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e73ca7-65c8-44e1-a4d0-dc01610440f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:59.027191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e73ca7-65c8-44e1-a4d0-dc01610440f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:59.034847 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab00786-d05f-457f-a83b-f8c54c36a951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:59.035564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00786-d05f-457f-a83b-f8c54c36a951 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:59.035842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00786-d05f-457f-a83b-f8c54c36a951 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:59.036110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00786-d05f-457f-a83b-f8c54c36a951 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:59.036751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab00786-d05f-457f-a83b-f8c54c36a951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:59.037065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab00786-d05f-457f-a83b-f8c54c36a951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:59.038438 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab00786-d05f-457f-a83b-f8c54c36a951 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:59.039365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e73ca7-65c8-44e1-a4d0-dc01610440f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:59.040862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e73ca7-65c8-44e1-a4d0-dc01610440f2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:59.050304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 292/591] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:59.051150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab00786-d05f-457f-a83b-f8c54c36a951 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d9e73ca7-65c8-44e1-a4d0-dc01610440f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:01:59.060292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:59.061505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:01:59.062632 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:01:59.062632 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:01:59.062632 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:59.067243 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 300/592] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:59.067243 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e73ca7-65c8-44e1-a4d0-dc01610440f2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:59.093303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:59.167489 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab00786-d05f-457f-a83b-f8c54c36a951 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:59.175664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e73ca7-65c8-44e1-a4d0-dc01610440f2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:59.203021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecc5ce-7a7e-4fe9-9469-23ace9087453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:59.203700 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc5ce-7a7e-4fe9-9469-23ace9087453 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:59.203973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc5ce-7a7e-4fe9-9469-23ace9087453 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:59.204439 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc5ce-7a7e-4fe9-9469-23ace9087453 None admin] PATH_INFO: `/v3/projects/2b842ac074b44e148c15882a7964aa13/users/441575b8b496457b905e9d5f5846e092/roles/6e17076d694e40c29dd5e2dbc6998270` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:59.212442 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd1249-9dfd-4f4d-ac07-87f9cd797545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:59.213077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd1249-9dfd-4f4d-ac07-87f9cd797545 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:59.213353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd1249-9dfd-4f4d-ac07-87f9cd797545 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:59.213631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd1249-9dfd-4f4d-ac07-87f9cd797545 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:59.220343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd1249-9dfd-4f4d-ac07-87f9cd797545 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:59.220484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecc5ce-7a7e-4fe9-9469-23ace9087453 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b842ac074b44e148c15882a7964aa13, user_id=441575b8b496457b905e9d5f5846e092, role_id=6e17076d694e40c29dd5e2dbc6998270)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:59.221817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd1249-9dfd-4f4d-ac07-87f9cd797545 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:59.222110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecc5ce-7a7e-4fe9-9469-23ace9087453 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d3cd1249-9dfd-4f4d-ac07-87f9cd797545 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:01:59.233638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:59.234586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:59.236141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:01:59.236141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:01:59.236141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:01:59.236141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:01:59.236141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:01:59.236141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:01:59.236141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:59.236141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:59.236141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 301/593] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:59.236141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd1249-9dfd-4f4d-ac07-87f9cd797545 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:59.251102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 293/594] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:01:59 2022] PUT /identity/v3/projects/2b842ac074b44e148c15882a7964aa13/users/441575b8b496457b905e9d5f5846e092/roles/6e17076d694e40c29dd5e2dbc6998270 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:01:59.251102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecc5ce-7a7e-4fe9-9469-23ace9087453 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:59.288776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:59.365457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd1249-9dfd-4f4d-ac07-87f9cd797545 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:59.383738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:59.427002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecc5ce-7a7e-4fe9-9469-23ace9087453 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:59.430720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc561e-69d8-4360-9902-d4e40ec7b5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:59.431687 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc561e-69d8-4360-9902-d4e40ec7b5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:59.432175 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc561e-69d8-4360-9902-d4e40ec7b5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:59.432648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc561e-69d8-4360-9902-d4e40ec7b5d7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:59.449434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc561e-69d8-4360-9902-d4e40ec7b5d7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:59.452281 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc561e-69d8-4360-9902-d4e40ec7b5d7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-56cc561e-69d8-4360-9902-d4e40ec7b5d7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:01:59.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:59.474524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:59.475554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:59.475554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:59.475554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:01:59.475554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:01:59.475554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:01:59.475554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:01:59.475554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:01:59.475554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:01:59.475554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 13:01:59.475554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:59.480036 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 302/595] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:01:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:59.480036 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc561e-69d8-4360-9902-d4e40ec7b5d7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:59.487586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e12cf2-0e6b-4c78-9618-c98d40dbf23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:59.488373 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e12cf2-0e6b-4c78-9618-c98d40dbf23f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:59.489478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e12cf2-0e6b-4c78-9618-c98d40dbf23f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:59.489478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e12cf2-0e6b-4c78-9618-c98d40dbf23f None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:59.489832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e12cf2-0e6b-4c78-9618-c98d40dbf23f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:59.490231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e12cf2-0e6b-4c78-9618-c98d40dbf23f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:59.493124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e12cf2-0e6b-4c78-9618-c98d40dbf23f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:59.507369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 294/596] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:59.508261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e12cf2-0e6b-4c78-9618-c98d40dbf23f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:59.632662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:59.637548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc561e-69d8-4360-9902-d4e40ec7b5d7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:59.658415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e12cf2-0e6b-4c78-9618-c98d40dbf23f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:59.680821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d93bf-7181-4b1a-8db9-4b01b76dd002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:59.681665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d93bf-7181-4b1a-8db9-4b01b76dd002 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:59.681952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d93bf-7181-4b1a-8db9-4b01b76dd002 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:59.682228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d93bf-7181-4b1a-8db9-4b01b76dd002 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:59.682927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d93bf-7181-4b1a-8db9-4b01b76dd002 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:59.683262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d93bf-7181-4b1a-8db9-4b01b76dd002 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:59.684739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d93bf-7181-4b1a-8db9-4b01b76dd002 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:59.698173 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 303/597] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:59.699017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d93bf-7181-4b1a-8db9-4b01b76dd002 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:59.706027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca6c85-973c-4908-af6c-ce0cba8784a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:59.707235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca6c85-973c-4908-af6c-ce0cba8784a1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:59.707658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca6c85-973c-4908-af6c-ce0cba8784a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:59.708132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca6c85-973c-4908-af6c-ce0cba8784a1 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:59.709004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca6c85-973c-4908-af6c-ce0cba8784a1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:01:59.709528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca6c85-973c-4908-af6c-ce0cba8784a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:59.711162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca6c85-973c-4908-af6c-ce0cba8784a1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:59.734536 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 295/598] 10.222.0.17 () {56 vars in 1139 bytes} [Fri Sep 9 13:01:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:01:59.736071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809f824-8756-46c2-aff0-13c2c88e914d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:59.736763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809f824-8756-46c2-aff0-13c2c88e914d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:59.737257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809f824-8756-46c2-aff0-13c2c88e914d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:59.743334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 296/599] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:01:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:01:59.744119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809f824-8756-46c2-aff0-13c2c88e914d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:59.748447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:59.818837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d93bf-7181-4b1a-8db9-4b01b76dd002 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:59.866950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809f824-8756-46c2-aff0-13c2c88e914d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:59.875726 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68245e-6205-4ba7-97b0-c868572442bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:59.876372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68245e-6205-4ba7-97b0-c868572442bd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:59.876653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68245e-6205-4ba7-97b0-c868572442bd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:59.877484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68245e-6205-4ba7-97b0-c868572442bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:59.890189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68245e-6205-4ba7-97b0-c868572442bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:59.892179 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68245e-6205-4ba7-97b0-c868572442bd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-4c68245e-6205-4ba7-97b0-c868572442bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:01:59.905519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:01:59.906596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:01:59.907617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:01:59.907617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:01:59.907617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:01:59.907617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:01:59.907617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:01:59.907617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:01:59.907617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:01:59.907617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:01:59.912396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 304/600] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:01:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:01:59.913204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68245e-6205-4ba7-97b0-c868572442bd None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:59.924528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04487b8c-8af4-4f9c-8023-977783eb5286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:59.926027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04487b8c-8af4-4f9c-8023-977783eb5286 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:59.926401 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04487b8c-8af4-4f9c-8023-977783eb5286 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:59.926767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04487b8c-8af4-4f9c-8023-977783eb5286 None admin] PATH_INFO: `/v3/projects/2b842ac074b44e148c15882a7964aa13/users/ea77fa8f5b754a2287f0639717f6e57a/roles/6e17076d694e40c29dd5e2dbc6998270` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:59.935708 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:59.944892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04487b8c-8af4-4f9c-8023-977783eb5286 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b842ac074b44e148c15882a7964aa13, user_id=ea77fa8f5b754a2287f0639717f6e57a, role_id=6e17076d694e40c29dd5e2dbc6998270)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:59.946354 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04487b8c-8af4-4f9c-8023-977783eb5286 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:59.959878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 297/601] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:01:59 2022] PUT /identity/v3/projects/2b842ac074b44e148c15882a7964aa13/users/ea77fa8f5b754a2287f0639717f6e57a/roles/6e17076d694e40c29dd5e2dbc6998270 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:01:59.961326 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e6154-b6c6-4ff1-8f2e-db9a4f03689a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:59.961709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e6154-b6c6-4ff1-8f2e-db9a4f03689a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:59.962083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e6154-b6c6-4ff1-8f2e-db9a4f03689a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:59.981132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8e2e6154-b6c6-4ff1-8f2e-db9a4f03689a None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:59.982528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8e2e6154-b6c6-4ff1-8f2e-db9a4f03689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:00.051742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68245e-6205-4ba7-97b0-c868572442bd None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:00.058794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:00.088568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 298/602] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:02:00.089886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e6154-b6c6-4ff1-8f2e-db9a4f03689a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:00.098739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a795b-08f9-46d9-abd3-89833487801b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:00.099733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a795b-08f9-46d9-abd3-89833487801b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:00.100248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a795b-08f9-46d9-abd3-89833487801b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:00.100732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a795b-08f9-46d9-abd3-89833487801b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:00.114744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a795b-08f9-46d9-abd3-89833487801b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:00.116709 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a795b-08f9-46d9-abd3-89833487801b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7f6a795b-08f9-46d9-abd3-89833487801b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:00.133684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:00.134721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:00.135684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:00.135684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:00.135684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:00.138085 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 305/603] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:00.144873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a795b-08f9-46d9-abd3-89833487801b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:00.214258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e6154-b6c6-4ff1-8f2e-db9a4f03689a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:00.263808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e31c6-772b-4ff6-80f5-a608f46a4bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:00.264438 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e31c6-772b-4ff6-80f5-a608f46a4bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:00.264717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e31c6-772b-4ff6-80f5-a608f46a4bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:00.264990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e31c6-772b-4ff6-80f5-a608f46a4bb8 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:00.265701 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e31c6-772b-4ff6-80f5-a608f46a4bb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:00.266008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e31c6-772b-4ff6-80f5-a608f46a4bb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:00.268492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e31c6-772b-4ff6-80f5-a608f46a4bb8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:00.271171 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:00.284772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 299/604] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:00.292212 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e31c6-772b-4ff6-80f5-a608f46a4bb8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:00.317494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a795b-08f9-46d9-abd3-89833487801b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:00.345615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:00.358822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea3c0d-a61b-4a1a-a523-0c1ca43d858d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:00.359572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea3c0d-a61b-4a1a-a523-0c1ca43d858d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:00.359951 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea3c0d-a61b-4a1a-a523-0c1ca43d858d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:00.360320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea3c0d-a61b-4a1a-a523-0c1ca43d858d None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:00.361081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea3c0d-a61b-4a1a-a523-0c1ca43d858d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:00.361484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea3c0d-a61b-4a1a-a523-0c1ca43d858d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:00.364000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea3c0d-a61b-4a1a-a523-0c1ca43d858d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:00.383196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 306/605] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:00.385592 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea3c0d-a61b-4a1a-a523-0c1ca43d858d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:00.423251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e31c6-772b-4ff6-80f5-a608f46a4bb8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:00.460832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f326ad-fc43-4779-905b-6f73b49c08a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:00.461817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f326ad-fc43-4779-905b-6f73b49c08a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:00.462242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f326ad-fc43-4779-905b-6f73b49c08a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:00.462676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f326ad-fc43-4779-905b-6f73b49c08a9 None admin] PATH_INFO: `/v3/projects/2b842ac074b44e148c15882a7964aa13/users/a7ef6cd64182474787134e3227beb3aa/roles/6e17076d694e40c29dd5e2dbc6998270` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:00.482125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f326ad-fc43-4779-905b-6f73b49c08a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b842ac074b44e148c15882a7964aa13, user_id=a7ef6cd64182474787134e3227beb3aa, role_id=6e17076d694e40c29dd5e2dbc6998270)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:00.483641 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f326ad-fc43-4779-905b-6f73b49c08a9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:00.512482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 300/606] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:02:00 2022] PUT /identity/v3/projects/2b842ac074b44e148c15882a7964aa13/users/a7ef6cd64182474787134e3227beb3aa/roles/6e17076d694e40c29dd5e2dbc6998270 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:00.520336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f326ad-fc43-4779-905b-6f73b49c08a9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:00.554417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea3c0d-a61b-4a1a-a523-0c1ca43d858d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:00.629053 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:00.638506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3511e-70ec-4677-9221-f92bc0633f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:00.639369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3511e-70ec-4677-9221-f92bc0633f61 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:00.641053 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3511e-70ec-4677-9221-f92bc0633f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:00.641053 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3511e-70ec-4677-9221-f92bc0633f61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:00.654515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3511e-70ec-4677-9221-f92bc0633f61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:00.657833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3511e-70ec-4677-9221-f92bc0633f61 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-cfd3511e-70ec-4677-9221-f92bc0633f61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:00.681381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:00.682537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:00.683711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:00.683711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:00.683711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:00.683711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:00.683711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:00.683711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:00.683711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:00.683711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:00.685365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 307/607] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:00.692849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3511e-70ec-4677-9221-f92bc0633f61 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:00.751225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:00.785132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f326ad-fc43-4779-905b-6f73b49c08a9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:00.829072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3511e-70ec-4677-9221-f92bc0633f61 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:00.869863 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fd177-172b-4cfc-bef0-5a3f464f75b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:00.870481 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fd177-172b-4cfc-bef0-5a3f464f75b1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:00.870824 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fd177-172b-4cfc-bef0-5a3f464f75b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:00.871135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fd177-172b-4cfc-bef0-5a3f464f75b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:00.872265 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fd177-172b-4cfc-bef0-5a3f464f75b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:00.872580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fd177-172b-4cfc-bef0-5a3f464f75b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:00.874151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fd177-172b-4cfc-bef0-5a3f464f75b1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:00.887083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 308/608] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:00.891813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fc411-795c-42dc-9b7c-582d11f89b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:00.892475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc411-795c-42dc-9b7c-582d11f89b4d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:00.892754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc411-795c-42dc-9b7c-582d11f89b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:00.893028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc411-795c-42dc-9b7c-582d11f89b4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:00.893683 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fd177-172b-4cfc-bef0-5a3f464f75b1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:00.934130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fc411-795c-42dc-9b7c-582d11f89b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-aa1fc411-795c-42dc-9b7c-582d11f89b4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:00.939253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:00.940887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:00.941939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:00.941939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:00.941939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:00.941939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:00.941939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:00.941939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:00.941939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:00.941939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:00.941939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:00.941939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:00.941939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:00.941939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:00.941939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 301/609] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:02:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:00.949860 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:00.952031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fc411-795c-42dc-9b7c-582d11f89b4d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:01.013665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fd177-172b-4cfc-bef0-5a3f464f75b1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:01.056017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edb474-58f2-43aa-9988-a55a95ff0254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:01.056812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edb474-58f2-43aa-9988-a55a95ff0254 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:01.056979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edb474-58f2-43aa-9988-a55a95ff0254 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:01.057250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edb474-58f2-43aa-9988-a55a95ff0254 None admin] PATH_INFO: `/v3/projects/2b842ac074b44e148c15882a7964aa13/users/e12deabe077b4095bfa7e2e8fac61d14/roles/6e17076d694e40c29dd5e2dbc6998270` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:01.074957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edb474-58f2-43aa-9988-a55a95ff0254 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b842ac074b44e148c15882a7964aa13, user_id=e12deabe077b4095bfa7e2e8fac61d14, role_id=6e17076d694e40c29dd5e2dbc6998270)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:01.077017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edb474-58f2-43aa-9988-a55a95ff0254 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:01.101210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 309/610] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:02:00 2022] PUT /identity/v3/projects/2b842ac074b44e148c15882a7964aa13/users/e12deabe077b4095bfa7e2e8fac61d14/roles/6e17076d694e40c29dd5e2dbc6998270 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:01.176257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fc411-795c-42dc-9b7c-582d11f89b4d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:01.227485 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e630cfe-1b6b-4e73-a0d8-be9fa8bcf0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:01.228287 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e630cfe-1b6b-4e73-a0d8-be9fa8bcf0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:01.228452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e630cfe-1b6b-4e73-a0d8-be9fa8bcf0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:01.228677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e630cfe-1b6b-4e73-a0d8-be9fa8bcf0c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:01.229417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e630cfe-1b6b-4e73-a0d8-be9fa8bcf0c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:01.229777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e630cfe-1b6b-4e73-a0d8-be9fa8bcf0c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:01.231241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e630cfe-1b6b-4e73-a0d8-be9fa8bcf0c0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:01.241113 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 302/611] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:02:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:01.246719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edb474-58f2-43aa-9988-a55a95ff0254 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:01.410289 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:01.415178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edb474-58f2-43aa-9988-a55a95ff0254 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:01.456142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f09d1-1435-4750-9445-55be28df0819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:01.456790 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f09d1-1435-4750-9445-55be28df0819 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:01.457068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f09d1-1435-4750-9445-55be28df0819 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:01.457337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f09d1-1435-4750-9445-55be28df0819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:01.467954 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f09d1-1435-4750-9445-55be28df0819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:01.469418 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f09d1-1435-4750-9445-55be28df0819 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-056f09d1-1435-4750-9445-55be28df0819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:01.486455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:01.488213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:01.489150 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:01.489150 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:01.489150 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:01.489150 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 310/612] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:01.495447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e630cfe-1b6b-4e73-a0d8-be9fa8bcf0c0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:01.516957 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:01.641437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e630cfe-1b6b-4e73-a0d8-be9fa8bcf0c0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:01.694148 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b9cd5-94e1-465b-a924-15d43d3e0bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:01.694791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b9cd5-94e1-465b-a924-15d43d3e0bec None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:01.695083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b9cd5-94e1-465b-a924-15d43d3e0bec None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:01.695366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b9cd5-94e1-465b-a924-15d43d3e0bec None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:01.696042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b9cd5-94e1-465b-a924-15d43d3e0bec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:01.696464 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b9cd5-94e1-465b-a924-15d43d3e0bec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:01.698003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b9cd5-94e1-465b-a924-15d43d3e0bec None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:01.715169 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 303/613] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:01.718204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db33c11-22b1-49be-b497-f205661aaffc None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:01.718496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db33c11-22b1-49be-b497-f205661aaffc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:01.718775 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db33c11-22b1-49be-b497-f205661aaffc None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:01.720129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 311/614] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:01.721751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b9cd5-94e1-465b-a924-15d43d3e0bec None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:01.742546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb02031-ff1a-48de-b5cc-0feed4a2b132 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:01.742840 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb02031-ff1a-48de-b5cc-0feed4a2b132 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:01.745831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb02031-ff1a-48de-b5cc-0feed4a2b132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:01.773519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4bb02031-ff1a-48de-b5cc-0feed4a2b132 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:01.780338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4bb02031-ff1a-48de-b5cc-0feed4a2b132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:01.850350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:01.880533 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b9cd5-94e1-465b-a924-15d43d3e0bec None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:01.890178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 312/615] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:02:01.934416 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f40cc-aadd-45f8-a8c6-78cd5bde2164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:01.935410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f40cc-aadd-45f8-a8c6-78cd5bde2164 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:01.935931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f40cc-aadd-45f8-a8c6-78cd5bde2164 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:01.936395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f40cc-aadd-45f8-a8c6-78cd5bde2164 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:01.946025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f40cc-aadd-45f8-a8c6-78cd5bde2164 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:01.955071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f40cc-aadd-45f8-a8c6-78cd5bde2164 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-343f40cc-aadd-45f8-a8c6-78cd5bde2164 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:01.967965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:01.968966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:01.969938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:01.969938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:01.969938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:01.969938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:01.969938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:01.969938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:01.969938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:01.969938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:01.969938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 13:02:01.969938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:01.976233 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 304/616] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:02:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:01.981740 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb02031-ff1a-48de-b5cc-0feed4a2b132 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:02.075233 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:02.094152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb02031-ff1a-48de-b5cc-0feed4a2b132 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:02.131291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b1ffe-1786-4442-a17f-2cdb8cf841ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:02.131908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b1ffe-1786-4442-a17f-2cdb8cf841ea None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:02.132184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b1ffe-1786-4442-a17f-2cdb8cf841ea None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:02.132449 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b1ffe-1786-4442-a17f-2cdb8cf841ea None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:02.133172 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b1ffe-1786-4442-a17f-2cdb8cf841ea None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:02.133471 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b1ffe-1786-4442-a17f-2cdb8cf841ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:02.134894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b1ffe-1786-4442-a17f-2cdb8cf841ea None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:02.154037 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 313/617] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 9 13:02:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:02.160182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f40cc-aadd-45f8-a8c6-78cd5bde2164 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:02.255745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b1ffe-1786-4442-a17f-2cdb8cf841ea None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:02.337666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f40cc-aadd-45f8-a8c6-78cd5bde2164 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:02.398170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104c8ff-17c1-4194-955a-d862b4d6402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:02.398812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104c8ff-17c1-4194-955a-d862b4d6402d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:02.399110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104c8ff-17c1-4194-955a-d862b4d6402d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:02.399382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104c8ff-17c1-4194-955a-d862b4d6402d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:02.401542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:02.414117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104c8ff-17c1-4194-955a-d862b4d6402d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:02.415605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104c8ff-17c1-4194-955a-d862b4d6402d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2104c8ff-17c1-4194-955a-d862b4d6402d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:02.431757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:02.432720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:02.433605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:02.433605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:02.433605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:02.433605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 305/618] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:02.439853 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104c8ff-17c1-4194-955a-d862b4d6402d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:02.475330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b1ffe-1786-4442-a17f-2cdb8cf841ea None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:02.546875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf58bf-8cde-47e2-b247-389b14831f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:02.547995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf58bf-8cde-47e2-b247-389b14831f74 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:02.548488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf58bf-8cde-47e2-b247-389b14831f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:02.548971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf58bf-8cde-47e2-b247-389b14831f74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:02.567291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:02.572918 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf58bf-8cde-47e2-b247-389b14831f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e5bf58bf-8cde-47e2-b247-389b14831f74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:02.578980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:02.580203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:02.581836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:02.581836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:02.581836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:02.581836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:02.581836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:02.581836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:02.581836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:02.581836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:02.581836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:02.581836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:02.581836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:02.581836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:02.592835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 314/619] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:02:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:02.615230 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf58bf-8cde-47e2-b247-389b14831f74 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:02.645004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104c8ff-17c1-4194-955a-d862b4d6402d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:02.726729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51aa53d-822d-4eba-8987-ddd8501b135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:02.727387 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51aa53d-822d-4eba-8987-ddd8501b135a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:02.727667 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51aa53d-822d-4eba-8987-ddd8501b135a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:02.727948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51aa53d-822d-4eba-8987-ddd8501b135a None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:02.728602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51aa53d-822d-4eba-8987-ddd8501b135a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:02.728908 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51aa53d-822d-4eba-8987-ddd8501b135a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:02.730289 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51aa53d-822d-4eba-8987-ddd8501b135a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:02.763931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 306/620] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:02.769175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51aa53d-822d-4eba-8987-ddd8501b135a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:02.810522 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf58bf-8cde-47e2-b247-389b14831f74 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:02.826871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:02.855838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d6f5f-8598-4152-af95-1ab2a58eec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:02.856457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d6f5f-8598-4152-af95-1ab2a58eec28 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:02.856738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d6f5f-8598-4152-af95-1ab2a58eec28 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:02.857012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d6f5f-8598-4152-af95-1ab2a58eec28 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:02.857691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d6f5f-8598-4152-af95-1ab2a58eec28 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:02.857996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d6f5f-8598-4152-af95-1ab2a58eec28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:02.859830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d6f5f-8598-4152-af95-1ab2a58eec28 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:02.870674 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 315/621] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:02:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:02.876063 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bb14f-98bc-4cde-af51-dfcb773ad5e5 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:02.876348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bb14f-98bc-4cde-af51-dfcb773ad5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:02.876625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bb14f-98bc-4cde-af51-dfcb773ad5e5 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:02.878584 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 316/622] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:02.880721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bb14f-98bc-4cde-af51-dfcb773ad5e5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:02.896357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51aa53d-822d-4eba-8987-ddd8501b135a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:02.937624 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c6b4a-e1a5-4b22-8668-c5a1d7359cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:02.938985 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c6b4a-e1a5-4b22-8668-c5a1d7359cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:02.939522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c6b4a-e1a5-4b22-8668-c5a1d7359cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:02.940009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c6b4a-e1a5-4b22-8668-c5a1d7359cc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:02.947552 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c6b4a-e1a5-4b22-8668-c5a1d7359cc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:02.949525 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c6b4a-e1a5-4b22-8668-c5a1d7359cc3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:02.951099 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-690c6b4a-e1a5-4b22-8668-c5a1d7359cc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:02.963886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:02.964845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:02.965736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:02.965736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:02.965736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:02.965736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:02.965736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:02.965736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:02.965736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:02.965736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:02.968215 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 307/623] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:02.969699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9460b3-5f18-40de-9174-50f3b0613198 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:02.970198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9460b3-5f18-40de-9174-50f3b0613198 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:02.970675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9460b3-5f18-40de-9174-50f3b0613198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:02.990777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6d9460b3-5f18-40de-9174-50f3b0613198 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:02.992579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6d9460b3-5f18-40de-9174-50f3b0613198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:02.993404 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bb14f-98bc-4cde-af51-dfcb773ad5e5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:03.030727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce7efa-8e6f-49d7-87a9-6b156cd4a707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:03.031484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce7efa-8e6f-49d7-87a9-6b156cd4a707 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.031967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce7efa-8e6f-49d7-87a9-6b156cd4a707 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.032146 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce7efa-8e6f-49d7-87a9-6b156cd4a707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.045466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce7efa-8e6f-49d7-87a9-6b156cd4a707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:03.046982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce7efa-8e6f-49d7-87a9-6b156cd4a707 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7fce7efa-8e6f-49d7-87a9-6b156cd4a707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:03.065383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:03.066431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:03.067367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:03.067367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:03.067367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:03.071224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 317/624] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:03.072103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce7efa-8e6f-49d7-87a9-6b156cd4a707 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:03.079133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 308/625] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:02:03.080320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9460b3-5f18-40de-9174-50f3b0613198 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:03.147831 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:03.182658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce7efa-8e6f-49d7-87a9-6b156cd4a707 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:03.198112 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:03.220390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250327c-41ec-4b17-b19e-fb3b33a61da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:03.221073 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250327c-41ec-4b17-b19e-fb3b33a61da4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.221508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250327c-41ec-4b17-b19e-fb3b33a61da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.221784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250327c-41ec-4b17-b19e-fb3b33a61da4 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.222328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9460b3-5f18-40de-9174-50f3b0613198 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:03.222519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250327c-41ec-4b17-b19e-fb3b33a61da4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:03.222781 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250327c-41ec-4b17-b19e-fb3b33a61da4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:03.224352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250327c-41ec-4b17-b19e-fb3b33a61da4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:03.238746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 318/626] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:03.240043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a50707-cc2c-47ae-9f4d-5a6f51e33c36 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.240315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a50707-cc2c-47ae-9f4d-5a6f51e33c36 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.240599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a50707-cc2c-47ae-9f4d-5a6f51e33c36 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.242127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 319/627] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:03.247869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a50707-cc2c-47ae-9f4d-5a6f51e33c36 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:03.262279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969d5d2-f04a-46eb-b28a-8772891efe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:03.262891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969d5d2-f04a-46eb-b28a-8772891efe4e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.263250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969d5d2-f04a-46eb-b28a-8772891efe4e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.263643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969d5d2-f04a-46eb-b28a-8772891efe4e None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.264288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969d5d2-f04a-46eb-b28a-8772891efe4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:03.264592 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969d5d2-f04a-46eb-b28a-8772891efe4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:03.265953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969d5d2-f04a-46eb-b28a-8772891efe4e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:03.278488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 309/628] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:03.279595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1beafa-d0eb-4c08-b9b7-8578504fdb93 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.279879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1beafa-d0eb-4c08-b9b7-8578504fdb93 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.280157 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1beafa-d0eb-4c08-b9b7-8578504fdb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.300226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cf1beafa-d0eb-4c08-b9b7-8578504fdb93 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:03.301551 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cf1beafa-d0eb-4c08-b9b7-8578504fdb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:03.381596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 310/629] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:02:03.382299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a50707-cc2c-47ae-9f4d-5a6f51e33c36 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:03.383655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1beafa-d0eb-4c08-b9b7-8578504fdb93 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:03.432107 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:03.444677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445dfe7-7f0d-44da-8ec8-c7ce36f0a143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:03.445735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445dfe7-7f0d-44da-8ec8-c7ce36f0a143 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.446212 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445dfe7-7f0d-44da-8ec8-c7ce36f0a143 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.447648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445dfe7-7f0d-44da-8ec8-c7ce36f0a143 None admin] PATH_INFO: `/v3/projects/2b842ac074b44e148c15882a7964aa13/users/c361a3e83f8e45c2b4d28f97f0a9d069/roles/6e17076d694e40c29dd5e2dbc6998270` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.475288 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5445dfe7-7f0d-44da-8ec8-c7ce36f0a143 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b842ac074b44e148c15882a7964aa13, user_id=c361a3e83f8e45c2b4d28f97f0a9d069, role_id=6e17076d694e40c29dd5e2dbc6998270)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:03.477799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5445dfe7-7f0d-44da-8ec8-c7ce36f0a143 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:03.500007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 320/630] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:02:03 2022] PUT /identity/v3/projects/2b842ac074b44e148c15882a7964aa13/users/c361a3e83f8e45c2b4d28f97f0a9d069/roles/6e17076d694e40c29dd5e2dbc6998270 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:03.501757 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445dfe7-7f0d-44da-8ec8-c7ce36f0a143 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:03.551109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:03.574071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1beafa-d0eb-4c08-b9b7-8578504fdb93 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:03.644964 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e49b80-2bdb-41ca-87fb-4903708eb738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:03.646552 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e49b80-2bdb-41ca-87fb-4903708eb738 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.647356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e49b80-2bdb-41ca-87fb-4903708eb738 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.648086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e49b80-2bdb-41ca-87fb-4903708eb738 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.663209 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e49b80-2bdb-41ca-87fb-4903708eb738 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:03.666411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e49b80-2bdb-41ca-87fb-4903708eb738 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:03.703121 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445dfe7-7f0d-44da-8ec8-c7ce36f0a143 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a2e49b80-2bdb-41ca-87fb-4903708eb738 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:03.704653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:03.706963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:03.708899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:03.708899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:03.708899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:03.708899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:03.708899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:03.708899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:03.708899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:03.708899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:03.708899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:02:03.708899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:03.712647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 311/631] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 13:02:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:03.714783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e49b80-2bdb-41ca-87fb-4903708eb738 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:03.759643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e4104-8fa8-4a39-8e92-e42a4a5f5a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:03.760478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e4104-8fa8-4a39-8e92-e42a4a5f5a8e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.760892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e4104-8fa8-4a39-8e92-e42a4a5f5a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.761292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e4104-8fa8-4a39-8e92-e42a4a5f5a8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.781280 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e4104-8fa8-4a39-8e92-e42a4a5f5a8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-dd2e4104-8fa8-4a39-8e92-e42a4a5f5a8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:03.789224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:03.790270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:03.791160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:03.791160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:03.791160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:03.791160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:03.791160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:03.791160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:03.791160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:03.791160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:03.791160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:03.791160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:03.791160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:03.791160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:03.794875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 321/632] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:02:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:03.796749 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e4104-8fa8-4a39-8e92-e42a4a5f5a8e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:03.836303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:03.855293 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e49b80-2bdb-41ca-87fb-4903708eb738 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:03.912788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:03.916180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da0031-e218-4036-9df0-3eb1b06b534f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:03.916776 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0031-e218-4036-9df0-3eb1b06b534f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.917053 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0031-e218-4036-9df0-3eb1b06b534f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.917327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0031-e218-4036-9df0-3eb1b06b534f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.928572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da0031-e218-4036-9df0-3eb1b06b534f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:03.930565 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e4104-8fa8-4a39-8e92-e42a4a5f5a8e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-36da0031-e218-4036-9df0-3eb1b06b534f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:03.933814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:03.934742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:03.935926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:03.935926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:03.935926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:03.935926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:03.935926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:03.935926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:03.935926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:03.935926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:03.935926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:03.935926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:03.935926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:03.935926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:03.935926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 312/633] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:02:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:03.941587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da0031-e218-4036-9df0-3eb1b06b534f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:03.967985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a011da-8ea9-40ec-b11c-7b8b847ba1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:03.968603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a011da-8ea9-40ec-b11c-7b8b847ba1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.968925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a011da-8ea9-40ec-b11c-7b8b847ba1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.969200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a011da-8ea9-40ec-b11c-7b8b847ba1b4 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.969933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a011da-8ea9-40ec-b11c-7b8b847ba1b4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:03.970239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a011da-8ea9-40ec-b11c-7b8b847ba1b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:03.971730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a011da-8ea9-40ec-b11c-7b8b847ba1b4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:03.994492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 322/634] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 13:02:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:03.994492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf257d-7248-4c34-a26f-33f94c9b7960 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.994492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf257d-7248-4c34-a26f-33f94c9b7960 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.994492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf257d-7248-4c34-a26f-33f94c9b7960 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.994492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 323/635] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:03.994492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf257d-7248-4c34-a26f-33f94c9b7960 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:04.050271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da0031-e218-4036-9df0-3eb1b06b534f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:04.087829 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9413536-3d02-4b1b-b10c-f60f9cf718a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:04.088442 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9413536-3d02-4b1b-b10c-f60f9cf718a2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:04.088723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9413536-3d02-4b1b-b10c-f60f9cf718a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:04.088999 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9413536-3d02-4b1b-b10c-f60f9cf718a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:04.089672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9413536-3d02-4b1b-b10c-f60f9cf718a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:04.089977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9413536-3d02-4b1b-b10c-f60f9cf718a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:04.091375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9413536-3d02-4b1b-b10c-f60f9cf718a2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:04.104840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 313/636] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:02:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:04.105529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9413536-3d02-4b1b-b10c-f60f9cf718a2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:04.110833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf257d-7248-4c34-a26f-33f94c9b7960 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:04.117572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:04.148905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c01a7-9903-4a95-943d-8891d1549ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:04.149515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c01a7-9903-4a95-943d-8891d1549ecb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:04.149791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c01a7-9903-4a95-943d-8891d1549ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:04.151157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c01a7-9903-4a95-943d-8891d1549ecb None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:04.151157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c01a7-9903-4a95-943d-8891d1549ecb None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:04.151157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c01a7-9903-4a95-943d-8891d1549ecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:04.152369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c01a7-9903-4a95-943d-8891d1549ecb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:04.164678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 324/637] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:02:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:04.164678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdaff7-d8e0-47a0-97f8-61c32fce0cd6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:04.164678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdaff7-d8e0-47a0-97f8-61c32fce0cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:04.164678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdaff7-d8e0-47a0-97f8-61c32fce0cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:04.171214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:04.183407 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2ffdaff7-d8e0-47a0-97f8-61c32fce0cd6 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:04.185562 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2ffdaff7-d8e0-47a0-97f8-61c32fce0cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:04.217009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9413536-3d02-4b1b-b10c-f60f9cf718a2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:04.251841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee21df-e3f4-463e-8a9d-9281ebdff41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:04.253128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee21df-e3f4-463e-8a9d-9281ebdff41b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:04.253453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee21df-e3f4-463e-8a9d-9281ebdff41b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:04.253807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee21df-e3f4-463e-8a9d-9281ebdff41b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:04.258453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 325/638] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:02:04.259648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffdaff7-d8e0-47a0-97f8-61c32fce0cd6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:04.341428 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee21df-e3f4-463e-8a9d-9281ebdff41b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:04.343692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee21df-e3f4-463e-8a9d-9281ebdff41b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:04.358698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:04.371892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffdaff7-d8e0-47a0-97f8-61c32fce0cd6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:04.407615 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf52a4-c93c-4a67-b969-bef82ec3c4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:04.409313 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf52a4-c93c-4a67-b969-bef82ec3c4fa None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:04.409760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf52a4-c93c-4a67-b969-bef82ec3c4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:04.410185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf52a4-c93c-4a67-b969-bef82ec3c4fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:04.412347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:04.423144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf52a4-c93c-4a67-b969-bef82ec3c4fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:04.425362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf52a4-c93c-4a67-b969-bef82ec3c4fa None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-bcaf52a4-c93c-4a67-b969-bef82ec3c4fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:04.441330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:04.442551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:04.443793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:04.443793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:04.443793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:04.445547 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 326/639] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:04.446554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf52a4-c93c-4a67-b969-bef82ec3c4fa None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:04.463469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 314/640] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:02:04.464565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee21df-e3f4-463e-8a9d-9281ebdff41b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:04.573234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf52a4-c93c-4a67-b969-bef82ec3c4fa None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:04.587374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee21df-e3f4-463e-8a9d-9281ebdff41b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:04.598767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:04.621324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16a268-b07d-4a06-b89a-5d5458c56437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:04.622120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16a268-b07d-4a06-b89a-5d5458c56437 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:04.622717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16a268-b07d-4a06-b89a-5d5458c56437 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:04.622997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16a268-b07d-4a06-b89a-5d5458c56437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:04.627295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f725ed-7756-44b1-ab18-773211da093e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:04.627910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f725ed-7756-44b1-ab18-773211da093e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:04.628189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f725ed-7756-44b1-ab18-773211da093e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:04.628461 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f725ed-7756-44b1-ab18-773211da093e None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:04.629100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f725ed-7756-44b1-ab18-773211da093e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:04.629403 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f725ed-7756-44b1-ab18-773211da093e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:04.630900 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f725ed-7756-44b1-ab18-773211da093e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:04.633579 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16a268-b07d-4a06-b89a-5d5458c56437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:04.635016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16a268-b07d-4a06-b89a-5d5458c56437 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:04.645480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 315/641] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:04.646523 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f725ed-7756-44b1-ab18-773211da093e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7a16a268-b07d-4a06-b89a-5d5458c56437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:04.649618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:04.650540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:04.651472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:04.651472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:04.651472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:04.655168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 327/642] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:04.655999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16a268-b07d-4a06-b89a-5d5458c56437 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:04.699802 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:04.756155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f725ed-7756-44b1-ab18-773211da093e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:04.767358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16a268-b07d-4a06-b89a-5d5458c56437 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:04.791480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b7c23-2406-4f93-8188-0a50a2adf5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:04.792118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b7c23-2406-4f93-8188-0a50a2adf5ae None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:04.792406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b7c23-2406-4f93-8188-0a50a2adf5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:04.792678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b7c23-2406-4f93-8188-0a50a2adf5ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:04.803922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b7c23-2406-4f93-8188-0a50a2adf5ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:04.804264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b25673-7483-4f3c-b26c-f0e5910fb484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:04.804900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b25673-7483-4f3c-b26c-f0e5910fb484 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:04.805185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b25673-7483-4f3c-b26c-f0e5910fb484 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:04.805381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b7c23-2406-4f93-8188-0a50a2adf5ae None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:04.805502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b25673-7483-4f3c-b26c-f0e5910fb484 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:04.816480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b25673-7483-4f3c-b26c-f0e5910fb484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a59b7c23-2406-4f93-8188-0a50a2adf5ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:04.820420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:04.821395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:04.822300 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:04.822300 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:04.822300 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-f3b25673-7483-4f3c-b26c-f0e5910fb484 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:04.822473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:04.823338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:04.824248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:04.824248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:04.824248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:04.824248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:04.824248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:04.824248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:04.824248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:04.824248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:04.824248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:04.824248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:04.824248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:04.824248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:04.824248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 328/643] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:02:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:04.826750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b25673-7483-4f3c-b26c-f0e5910fb484 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:04.827033 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 316/644] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:04.828450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b7c23-2406-4f93-8188-0a50a2adf5ae None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:04.854543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:04.930000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:04.934932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b25673-7483-4f3c-b26c-f0e5910fb484 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:04.942312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b7c23-2406-4f93-8188-0a50a2adf5ae None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:04.975259 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef5bba-2a8b-4944-b0e3-ece5c65fa834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:04.975819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef5bba-2a8b-4944-b0e3-ece5c65fa834 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:04.976156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef5bba-2a8b-4944-b0e3-ece5c65fa834 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:04.976434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef5bba-2a8b-4944-b0e3-ece5c65fa834 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:04.978068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e29b57-4153-4d19-a8fc-c5ab3b7dd20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:04.978839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e29b57-4153-4d19-a8fc-c5ab3b7dd20c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:04.979661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e29b57-4153-4d19-a8fc-c5ab3b7dd20c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:04.979942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e29b57-4153-4d19-a8fc-c5ab3b7dd20c None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:04.980587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e29b57-4153-4d19-a8fc-c5ab3b7dd20c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:04.980892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e29b57-4153-4d19-a8fc-c5ab3b7dd20c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:04.982269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e29b57-4153-4d19-a8fc-c5ab3b7dd20c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:04.984058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef5bba-2a8b-4944-b0e3-ece5c65fa834 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:04.985562 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef5bba-2a8b-4944-b0e3-ece5c65fa834 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:04.999491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 317/645] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:04.999491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e29b57-4153-4d19-a8fc-c5ab3b7dd20c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-35ef5bba-2a8b-4944-b0e3-ece5c65fa834 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:04.999718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:05.000606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:05.001545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:05.001545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:05.001545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:05.001545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:05.001545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:05.001545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:05.001545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:05.001545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:05.001545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 13:02:05.001545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:05.001545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 329/646] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:02:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:05.001545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef5bba-2a8b-4944-b0e3-ece5c65fa834 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:05.091884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:05.112997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef5bba-2a8b-4944-b0e3-ece5c65fa834 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:05.113170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e29b57-4153-4d19-a8fc-c5ab3b7dd20c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:05.149801 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff978dce-80ab-4df5-9efc-ac61a63e45fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:05.150393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ca7ae-7e5a-4869-9d46-5c74896faf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:05.150512 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff978dce-80ab-4df5-9efc-ac61a63e45fc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:05.150672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff978dce-80ab-4df5-9efc-ac61a63e45fc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:05.150946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff978dce-80ab-4df5-9efc-ac61a63e45fc None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:05.151041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ca7ae-7e5a-4869-9d46-5c74896faf73 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:05.151344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ca7ae-7e5a-4869-9d46-5c74896faf73 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:05.151569 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ca7ae-7e5a-4869-9d46-5c74896faf73 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:05.151676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff978dce-80ab-4df5-9efc-ac61a63e45fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:05.151914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff978dce-80ab-4df5-9efc-ac61a63e45fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:05.152226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ca7ae-7e5a-4869-9d46-5c74896faf73 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:05.152532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ca7ae-7e5a-4869-9d46-5c74896faf73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:05.153285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff978dce-80ab-4df5-9efc-ac61a63e45fc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:05.153881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ca7ae-7e5a-4869-9d46-5c74896faf73 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:05.162669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 330/647] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:02:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:05.163339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ca7ae-7e5a-4869-9d46-5c74896faf73 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:05.173831 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 318/648] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:05.173831 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff978dce-80ab-4df5-9efc-ac61a63e45fc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:05.188246 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:05.273278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ca7ae-7e5a-4869-9d46-5c74896faf73 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:05.275672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff978dce-80ab-4df5-9efc-ac61a63e45fc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:05.308124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79491c-ea7d-4a9a-b3da-52c1a4714674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:05.308765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79491c-ea7d-4a9a-b3da-52c1a4714674 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:05.309045 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79491c-ea7d-4a9a-b3da-52c1a4714674 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:05.309318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79491c-ea7d-4a9a-b3da-52c1a4714674 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:05.310975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda8076-a934-4639-ae65-827e2ecf64d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:05.311618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda8076-a934-4639-ae65-827e2ecf64d1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:05.311901 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda8076-a934-4639-ae65-827e2ecf64d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:05.312179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda8076-a934-4639-ae65-827e2ecf64d1 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:05.313166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda8076-a934-4639-ae65-827e2ecf64d1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:05.313474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda8076-a934-4639-ae65-827e2ecf64d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:05.315767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda8076-a934-4639-ae65-827e2ecf64d1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:05.318342 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79491c-ea7d-4a9a-b3da-52c1a4714674 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:05.319813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79491c-ea7d-4a9a-b3da-52c1a4714674 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:05.325428 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-cc79491c-ea7d-4a9a-b3da-52c1a4714674 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:05.332437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:05.333395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:05.334365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:05.334365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:05.334365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:05.334365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:05.334365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:05.334365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:05.334365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:05.334365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:05.334365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 13:02:05.334365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:05.334365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 331/649] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:02:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:05.334972 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79491c-ea7d-4a9a-b3da-52c1a4714674 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:05.340926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 319/650] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 9 13:02:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:05.341772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda8076-a934-4639-ae65-827e2ecf64d1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:05.428013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:05.449625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda8076-a934-4639-ae65-827e2ecf64d1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:05.452297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79491c-ea7d-4a9a-b3da-52c1a4714674 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:05.485268 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9da58e-f5d7-40e8-9875-c8e702126052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:05.485963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9da58e-f5d7-40e8-9875-c8e702126052 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:05.486766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9da58e-f5d7-40e8-9875-c8e702126052 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:05.487179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9da58e-f5d7-40e8-9875-c8e702126052 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:05.492110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d8f3b-19b9-4823-b844-21a8f6c42fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:05.492979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d8f3b-19b9-4823-b844-21a8f6c42fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:05.493462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d8f3b-19b9-4823-b844-21a8f6c42fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:05.493934 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d8f3b-19b9-4823-b844-21a8f6c42fa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:05.494057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9da58e-f5d7-40e8-9875-c8e702126052 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:05.495542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9da58e-f5d7-40e8-9875-c8e702126052 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:05.506834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d8f3b-19b9-4823-b844-21a8f6c42fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-fd9da58e-f5d7-40e8-9875-c8e702126052 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:05.508527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:05.509410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:05.510297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:05.510297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:05.510297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:05.510297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:05.510297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:05.510297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:05.510297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:05.510297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:05.510297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 320/651] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:05.510823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9da58e-f5d7-40e8-9875-c8e702126052 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:05.511203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d8f3b-19b9-4823-b844-21a8f6c42fa9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-631d8f3b-19b9-4823-b844-21a8f6c42fa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:05.532356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:05.533580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:05.534788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:05.534788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:05.534788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:05.540921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 332/652] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:05.540921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d8f3b-19b9-4823-b844-21a8f6c42fa9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:05.582081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:05.628311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9da58e-f5d7-40e8-9875-c8e702126052 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:05.662306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d8f3b-19b9-4823-b844-21a8f6c42fa9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:05.663800 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fefa92-17d4-479b-87fb-2e4424b81357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:05.664409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fefa92-17d4-479b-87fb-2e4424b81357 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:05.664682 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fefa92-17d4-479b-87fb-2e4424b81357 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:05.665563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fefa92-17d4-479b-87fb-2e4424b81357 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:05.665684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fefa92-17d4-479b-87fb-2e4424b81357 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:05.665956 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fefa92-17d4-479b-87fb-2e4424b81357 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:05.667409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fefa92-17d4-479b-87fb-2e4424b81357 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:05.690843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 321/653] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 9 13:02:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:05.690843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fefa92-17d4-479b-87fb-2e4424b81357 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:05.702042 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:05.710874 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4bc20-5a9b-41ae-84b7-a8ab12558bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:05.711677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4bc20-5a9b-41ae-84b7-a8ab12558bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:05.711989 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4bc20-5a9b-41ae-84b7-a8ab12558bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:05.712254 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4bc20-5a9b-41ae-84b7-a8ab12558bf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:05.787950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4bc20-5a9b-41ae-84b7-a8ab12558bf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:05.789672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4bc20-5a9b-41ae-84b7-a8ab12558bf3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:05.814869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fefa92-17d4-479b-87fb-2e4424b81357 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:05.833303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:05.853848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90dbbb-e709-469e-b07f-dfde874826f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:05.854459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90dbbb-e709-469e-b07f-dfde874826f5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:05.854742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90dbbb-e709-469e-b07f-dfde874826f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:05.855022 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90dbbb-e709-469e-b07f-dfde874826f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:05.855739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90dbbb-e709-469e-b07f-dfde874826f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:05.856050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90dbbb-e709-469e-b07f-dfde874826f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:05.857482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90dbbb-e709-469e-b07f-dfde874826f5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:05.869423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 322/654] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:05.870390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90dbbb-e709-469e-b07f-dfde874826f5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:05.894837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 333/655] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:02:05.895806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4bc20-5a9b-41ae-84b7-a8ab12558bf3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:05.928064 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:05.972211 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90dbbb-e709-469e-b07f-dfde874826f5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:05.998039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4bc20-5a9b-41ae-84b7-a8ab12558bf3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:06.007100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552f2a8-0e77-4132-929a-6e3ce7490c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:06.007716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552f2a8-0e77-4132-929a-6e3ce7490c16 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:06.007994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552f2a8-0e77-4132-929a-6e3ce7490c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:06.008266 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552f2a8-0e77-4132-929a-6e3ce7490c16 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:06.008936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552f2a8-0e77-4132-929a-6e3ce7490c16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:06.009245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552f2a8-0e77-4132-929a-6e3ce7490c16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:06.010611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552f2a8-0e77-4132-929a-6e3ce7490c16 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:06.022077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 323/656] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:06.023269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c29ce6-8a22-4a18-b942-3a9cc7e781ba None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:06.023555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c29ce6-8a22-4a18-b942-3a9cc7e781ba None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:06.023834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c29ce6-8a22-4a18-b942-3a9cc7e781ba None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:06.025313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 324/657] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:06.026030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c29ce6-8a22-4a18-b942-3a9cc7e781ba None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:06.034802 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c802b5c-c7c9-4538-a22d-6ef68a37a992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:06.035449 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c802b5c-c7c9-4538-a22d-6ef68a37a992 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:06.035727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c802b5c-c7c9-4538-a22d-6ef68a37a992 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:06.035999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c802b5c-c7c9-4538-a22d-6ef68a37a992 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:06.061718 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:06.105341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c802b5c-c7c9-4538-a22d-6ef68a37a992 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:06.106815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c802b5c-c7c9-4538-a22d-6ef68a37a992 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:06.129222 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c29ce6-8a22-4a18-b942-3a9cc7e781ba None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:06.136408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:06.163261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c08a0d-2341-4165-83b4-35e56340a113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:06.163867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c08a0d-2341-4165-83b4-35e56340a113 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:06.164143 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c08a0d-2341-4165-83b4-35e56340a113 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:06.164414 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c08a0d-2341-4165-83b4-35e56340a113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:06.165130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c08a0d-2341-4165-83b4-35e56340a113 None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=441575b8b496457b905e9d5f5846e092) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:06.165437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c08a0d-2341-4165-83b4-35e56340a113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:06.166860 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c08a0d-2341-4165-83b4-35e56340a113 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:06.172635 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 325/658] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:06 2022] GET /identity/v3/role_assignments?user.id=441575b8b496457b905e9d5f5846e092&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:06.173405 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c08a0d-2341-4165-83b4-35e56340a113 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:06.222010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 334/659] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:02:06.223546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c802b5c-c7c9-4538-a22d-6ef68a37a992 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:06.288783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:06.292022 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c08a0d-2341-4165-83b4-35e56340a113 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:06.333092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136743dd-f739-43ee-a425-b20e326c3add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:06.333661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136743dd-f739-43ee-a425-b20e326c3add None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:06.333941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136743dd-f739-43ee-a425-b20e326c3add None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:06.334210 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136743dd-f739-43ee-a425-b20e326c3add None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:06.341554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c802b5c-c7c9-4538-a22d-6ef68a37a992 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:06.346696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136743dd-f739-43ee-a425-b20e326c3add None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:06.348111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136743dd-f739-43ee-a425-b20e326c3add None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-136743dd-f739-43ee-a425-b20e326c3add None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:06.369811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:06.370816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:06.371786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:06.371786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:06.371786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:06.371786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 326/660] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:06.372304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f716315-0bb7-4e10-8a40-e22f962a158b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:06.372574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f716315-0bb7-4e10-8a40-e22f962a158b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:06.372812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f716315-0bb7-4e10-8a40-e22f962a158b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:06.381847 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:06.387792 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e78fc-3988-48df-a394-e815c94b555b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:06.388427 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e78fc-3988-48df-a394-e815c94b555b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:06.388701 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e78fc-3988-48df-a394-e815c94b555b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:06.388968 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e78fc-3988-48df-a394-e815c94b555b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:06.392559 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0f716315-0bb7-4e10-8a40-e22f962a158b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:06.393968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0f716315-0bb7-4e10-8a40-e22f962a158b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:06.396588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e78fc-3988-48df-a394-e815c94b555b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:06.398016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e78fc-3988-48df-a394-e815c94b555b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-946e78fc-3988-48df-a394-e815c94b555b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:06.412019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:06.412997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:06.416292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:06.416292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:06.416292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:06.416292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:06.416292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:06.416292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:06.416292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:06.416292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:06.416292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 13:02:06.416292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:06.416292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 335/661] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:02:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:06.416292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e78fc-3988-48df-a394-e815c94b555b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:06.473524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 327/662] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:02:06.479071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f716315-0bb7-4e10-8a40-e22f962a158b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:06.529158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e78fc-3988-48df-a394-e815c94b555b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:06.566344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7d971-e436-441a-9a78-506889bf141a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:06.567087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7d971-e436-441a-9a78-506889bf141a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:06.567481 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7d971-e436-441a-9a78-506889bf141a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:06.567784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7d971-e436-441a-9a78-506889bf141a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:06.571865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:06.582035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7d971-e436-441a-9a78-506889bf141a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:06.583704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7d971-e436-441a-9a78-506889bf141a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:06.596588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f716315-0bb7-4e10-8a40-e22f962a158b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0ce7d971-e436-441a-9a78-506889bf141a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:06.604564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:06.605639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:06.606526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:06.606526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:06.606526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:06.609799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 336/663] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:06.609799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7d971-e436-441a-9a78-506889bf141a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:06.644170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13c13b-0262-4597-96a5-0a1e62595293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:06.645089 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13c13b-0262-4597-96a5-0a1e62595293 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:06.645702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13c13b-0262-4597-96a5-0a1e62595293 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:06.646535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13c13b-0262-4597-96a5-0a1e62595293 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:06.647632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13c13b-0262-4597-96a5-0a1e62595293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:06.648090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13c13b-0262-4597-96a5-0a1e62595293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:06.649893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13c13b-0262-4597-96a5-0a1e62595293 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:06.664715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:06.667759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 328/664] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:06.669083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13c13b-0262-4597-96a5-0a1e62595293 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:06.729624 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7d971-e436-441a-9a78-506889bf141a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:06.771191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4830e52-6cc5-4c86-a609-be03be25f22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:06.771909 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4830e52-6cc5-4c86-a609-be03be25f22c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:06.772255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4830e52-6cc5-4c86-a609-be03be25f22c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:06.772594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4830e52-6cc5-4c86-a609-be03be25f22c None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:06.773399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4830e52-6cc5-4c86-a609-be03be25f22c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:06.773793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4830e52-6cc5-4c86-a609-be03be25f22c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:06.775766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4830e52-6cc5-4c86-a609-be03be25f22c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:06.794435 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 337/665] 10.222.0.17 () {56 vars in 1139 bytes} [Fri Sep 9 13:02:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:06.803091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4830e52-6cc5-4c86-a609-be03be25f22c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:06.803250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13c13b-0262-4597-96a5-0a1e62595293 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:06.840470 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5376e8b-4351-4d06-a21f-b22852e8b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:06.841871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5376e8b-4351-4d06-a21f-b22852e8b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:06.842191 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5376e8b-4351-4d06-a21f-b22852e8b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:06.842479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5376e8b-4351-4d06-a21f-b22852e8b5a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:06.843159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5376e8b-4351-4d06-a21f-b22852e8b5a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:06.843468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5376e8b-4351-4d06-a21f-b22852e8b5a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:06.844830 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5376e8b-4351-4d06-a21f-b22852e8b5a6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:06.865459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 329/666] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:06.865459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5376e8b-4351-4d06-a21f-b22852e8b5a6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:06.868922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:06.911779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4830e52-6cc5-4c86-a609-be03be25f22c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:06.913619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:06.954044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46676905-817f-448d-8db2-af706310dc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:06.954985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46676905-817f-448d-8db2-af706310dc97 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:06.955164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46676905-817f-448d-8db2-af706310dc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:06.955492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46676905-817f-448d-8db2-af706310dc97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:06.962501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46676905-817f-448d-8db2-af706310dc97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:06.964006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46676905-817f-448d-8db2-af706310dc97 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-46676905-817f-448d-8db2-af706310dc97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:06.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:06.977261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:06.978165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:06.978165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:06.978165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:06.978165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:06.978165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:06.978165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:06.978165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:06.978165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:06.978165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 338/667] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:06.978165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46676905-817f-448d-8db2-af706310dc97 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:06.991622 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5376e8b-4351-4d06-a21f-b22852e8b5a6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:07.027606 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd1b4b-d0e5-425a-a645-811625334075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:07.028233 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd1b4b-d0e5-425a-a645-811625334075 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:07.028506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd1b4b-d0e5-425a-a645-811625334075 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:07.028770 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd1b4b-d0e5-425a-a645-811625334075 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:07.039127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd1b4b-d0e5-425a-a645-811625334075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-17bd1b4b-d0e5-425a-a645-811625334075 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:07.044115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:07.045019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:07.045891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:07.045891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:07.045891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:07.045891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:07.045891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:07.045891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:07.045891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:07.045891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:07.045891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:07.045891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:07.045891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:07.045891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:07.045891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 330/668] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:02:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:07.046588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd1b4b-d0e5-425a-a645-811625334075 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:07.095846 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46676905-817f-448d-8db2-af706310dc97 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:07.108725 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:07.131982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cf9c2-be4b-492f-9e02-b18358f8c15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:07.132598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cf9c2-be4b-492f-9e02-b18358f8c15c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:07.132877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cf9c2-be4b-492f-9e02-b18358f8c15c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:07.133146 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cf9c2-be4b-492f-9e02-b18358f8c15c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:07.152105 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd1b4b-d0e5-425a-a645-811625334075 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:07.168068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:07.190184 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f1453-ecf7-422b-a3d7-354fb559d757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:07.190816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f1453-ecf7-422b-a3d7-354fb559d757 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:07.191124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f1453-ecf7-422b-a3d7-354fb559d757 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:07.191399 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f1453-ecf7-422b-a3d7-354fb559d757 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:07.198059 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f1453-ecf7-422b-a3d7-354fb559d757 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:07.199543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f1453-ecf7-422b-a3d7-354fb559d757 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:07.209374 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7cf9c2-be4b-492f-9e02-b18358f8c15c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:07.210840 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7cf9c2-be4b-492f-9e02-b18358f8c15c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4e9f1453-ecf7-422b-a3d7-354fb559d757 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:07.210944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:07.211810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:07.212713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:07.212713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:07.212713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:07.212713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:07.212713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:07.212713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:07.212713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:07.212713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:07.212713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 331/669] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:07.213201 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f1453-ecf7-422b-a3d7-354fb559d757 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:07.324530 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 339/670] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:02:07.325373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cf9c2-be4b-492f-9e02-b18358f8c15c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:07.325654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f1453-ecf7-422b-a3d7-354fb559d757 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:07.352951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:07.366542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb500cfe-4947-436a-b327-781276baaa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:07.367185 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb500cfe-4947-436a-b327-781276baaa5a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:07.367464 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb500cfe-4947-436a-b327-781276baaa5a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:07.367734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb500cfe-4947-436a-b327-781276baaa5a None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:07.368424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb500cfe-4947-436a-b327-781276baaa5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:07.368730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb500cfe-4947-436a-b327-781276baaa5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:07.370578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb500cfe-4947-436a-b327-781276baaa5a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:07.383806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 332/671] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:07.385378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb500cfe-4947-436a-b327-781276baaa5a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:07.395188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:07.437891 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cf9c2-be4b-492f-9e02-b18358f8c15c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:07.475038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774d713-6e5e-4ebc-9f94-910e8e0a2f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:07.475668 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774d713-6e5e-4ebc-9f94-910e8e0a2f80 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:07.475936 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774d713-6e5e-4ebc-9f94-910e8e0a2f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:07.476209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774d713-6e5e-4ebc-9f94-910e8e0a2f80 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:07.476865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774d713-6e5e-4ebc-9f94-910e8e0a2f80 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:07.477166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774d713-6e5e-4ebc-9f94-910e8e0a2f80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:07.478518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774d713-6e5e-4ebc-9f94-910e8e0a2f80 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:07.487703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 340/672] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:02:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:07.492823 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774d713-6e5e-4ebc-9f94-910e8e0a2f80 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:07.496182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb500cfe-4947-436a-b327-781276baaa5a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:07.532071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7205833-5498-42b3-9df2-be39f1a371ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:07.532686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7205833-5498-42b3-9df2-be39f1a371ac None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:07.533569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7205833-5498-42b3-9df2-be39f1a371ac None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:07.533961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7205833-5498-42b3-9df2-be39f1a371ac None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:07.534776 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7205833-5498-42b3-9df2-be39f1a371ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:07.535111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7205833-5498-42b3-9df2-be39f1a371ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:07.536553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7205833-5498-42b3-9df2-be39f1a371ac None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:07.550494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 333/673] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:07.551342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7205833-5498-42b3-9df2-be39f1a371ac None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:07.597596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:07.619477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774d713-6e5e-4ebc-9f94-910e8e0a2f80 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:07.645534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:07.658273 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc99df5-ae2d-4e06-91af-156973e30db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:07.658908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc99df5-ae2d-4e06-91af-156973e30db2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:07.659029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7205833-5498-42b3-9df2-be39f1a371ac None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:07.659220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc99df5-ae2d-4e06-91af-156973e30db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:07.659488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc99df5-ae2d-4e06-91af-156973e30db2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:07.670389 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc99df5-ae2d-4e06-91af-156973e30db2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:07.671974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc99df5-ae2d-4e06-91af-156973e30db2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-1cc99df5-ae2d-4e06-91af-156973e30db2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:07.687967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:07.688903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:07.690532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:07.690532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:07.690532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:07.690532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 341/674] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:07.690532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc99df5-ae2d-4e06-91af-156973e30db2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:07.704005 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64e789-fbc3-47b4-9b9e-51c88179305e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:07.705013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64e789-fbc3-47b4-9b9e-51c88179305e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:07.705285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64e789-fbc3-47b4-9b9e-51c88179305e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:07.705552 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64e789-fbc3-47b4-9b9e-51c88179305e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:07.706584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64e789-fbc3-47b4-9b9e-51c88179305e None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=ea77fa8f5b754a2287f0639717f6e57a) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:07.706895 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64e789-fbc3-47b4-9b9e-51c88179305e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:07.708377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64e789-fbc3-47b4-9b9e-51c88179305e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:07.714313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 334/675] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:07 2022] GET /identity/v3/role_assignments?user.id=ea77fa8f5b754a2287f0639717f6e57a&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:07.714921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64e789-fbc3-47b4-9b9e-51c88179305e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:07.835950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc99df5-ae2d-4e06-91af-156973e30db2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:07.852782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64e789-fbc3-47b4-9b9e-51c88179305e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:07.862467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:07.882070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef1619-58e5-4c66-93e2-0f0339d3e835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:07.883118 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef1619-58e5-4c66-93e2-0f0339d3e835 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:07.883675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef1619-58e5-4c66-93e2-0f0339d3e835 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:07.884194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef1619-58e5-4c66-93e2-0f0339d3e835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:07.900950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef1619-58e5-4c66-93e2-0f0339d3e835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:07.902754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef1619-58e5-4c66-93e2-0f0339d3e835 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:07.904176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d826d1-c68e-44ab-a640-fb6a744b8000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:07.904786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d826d1-c68e-44ab-a640-fb6a744b8000 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:07.905059 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d826d1-c68e-44ab-a640-fb6a744b8000 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:07.905486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d826d1-c68e-44ab-a640-fb6a744b8000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:07.906222 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d826d1-c68e-44ab-a640-fb6a744b8000 None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=a7ef6cd64182474787134e3227beb3aa) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:07.906835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d826d1-c68e-44ab-a640-fb6a744b8000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:07.908912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d826d1-c68e-44ab-a640-fb6a744b8000 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:07.916722 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 335/676] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:07 2022] GET /identity/v3/role_assignments?user.id=a7ef6cd64182474787134e3227beb3aa&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:07.917083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d826d1-c68e-44ab-a640-fb6a744b8000 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-14ef1619-58e5-4c66-93e2-0f0339d3e835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:07.924253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:07.925364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:07.926297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:07.926297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:07.926297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:07.929239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 342/677] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:07.936388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef1619-58e5-4c66-93e2-0f0339d3e835 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:07.976153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:08.054556 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d826d1-c68e-44ab-a640-fb6a744b8000 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:08.081453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef1619-58e5-4c66-93e2-0f0339d3e835 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:08.109503 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba95f87-2aff-4f39-ae39-cb6784a4d81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:08.111811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba95f87-2aff-4f39-ae39-cb6784a4d81b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:08.112746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba95f87-2aff-4f39-ae39-cb6784a4d81b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:08.113232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba95f87-2aff-4f39-ae39-cb6784a4d81b None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:08.114326 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba95f87-2aff-4f39-ae39-cb6784a4d81b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:08.115236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba95f87-2aff-4f39-ae39-cb6784a4d81b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:08.117014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba95f87-2aff-4f39-ae39-cb6784a4d81b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:08.130973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a900f-9a16-4663-82b9-8c413bb8e258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:08.131681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a900f-9a16-4663-82b9-8c413bb8e258 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:08.131961 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a900f-9a16-4663-82b9-8c413bb8e258 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:08.132234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a900f-9a16-4663-82b9-8c413bb8e258 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:08.132883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a900f-9a16-4663-82b9-8c413bb8e258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:08.133194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a900f-9a16-4663-82b9-8c413bb8e258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:08.134572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a900f-9a16-4663-82b9-8c413bb8e258 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:08.139310 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 336/678] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:08.142233 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba95f87-2aff-4f39-ae39-cb6784a4d81b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:08.151141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 343/679] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:08.156116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a900f-9a16-4663-82b9-8c413bb8e258 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:08.181220 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:08.264367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:08.280064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba95f87-2aff-4f39-ae39-cb6784a4d81b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:08.281589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a900f-9a16-4663-82b9-8c413bb8e258 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:08.320328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47d1b5-4b98-4988-ad25-aeea12508f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:08.321216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47d1b5-4b98-4988-ad25-aeea12508f14 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:08.321685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47d1b5-4b98-4988-ad25-aeea12508f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:08.322135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47d1b5-4b98-4988-ad25-aeea12508f14 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:08.332745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07a464-a2af-42fd-8147-32e4b77e9d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:08.333845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07a464-a2af-42fd-8147-32e4b77e9d29 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:08.334200 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07a464-a2af-42fd-8147-32e4b77e9d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:08.334584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07a464-a2af-42fd-8147-32e4b77e9d29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:08.335101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47d1b5-4b98-4988-ad25-aeea12508f14 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:08.337074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47d1b5-4b98-4988-ad25-aeea12508f14 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:08.343560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07a464-a2af-42fd-8147-32e4b77e9d29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:08.345652 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07a464-a2af-42fd-8147-32e4b77e9d29 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-da47d1b5-4b98-4988-ad25-aeea12508f14 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:08.350017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:08.350973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:08.352003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:08.352003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:08.352003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:08.352003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:08.352003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:08.352003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:08.352003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:08.352003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:08.352003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 13:02:08.352003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:08.354517 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 344/680] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:02:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6c07a464-a2af-42fd-8147-32e4b77e9d29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:08.358323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:08.359218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:08.360188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:08.360188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:08.360188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:08.360188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:08.360188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:08.360188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:08.360188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:08.360188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:08.362566 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 337/681] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:08.363774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47d1b5-4b98-4988-ad25-aeea12508f14 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:08.366182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07a464-a2af-42fd-8147-32e4b77e9d29 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:08.490698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47d1b5-4b98-4988-ad25-aeea12508f14 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:08.497828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:08.509377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07a464-a2af-42fd-8147-32e4b77e9d29 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:08.531244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a936c-c4c1-406d-8ab8-68a17b6fb554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:08.531867 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a936c-c4c1-406d-8ab8-68a17b6fb554 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:08.532142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a936c-c4c1-406d-8ab8-68a17b6fb554 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:08.532414 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a936c-c4c1-406d-8ab8-68a17b6fb554 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:08.534133 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a936c-c4c1-406d-8ab8-68a17b6fb554 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:08.534979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a936c-c4c1-406d-8ab8-68a17b6fb554 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:08.536760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a936c-c4c1-406d-8ab8-68a17b6fb554 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:08.551603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60b3e2-00e7-4018-814a-5f43bdb01a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:08.552225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60b3e2-00e7-4018-814a-5f43bdb01a4a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:08.553131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60b3e2-00e7-4018-814a-5f43bdb01a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:08.553468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60b3e2-00e7-4018-814a-5f43bdb01a4a None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:08.554461 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60b3e2-00e7-4018-814a-5f43bdb01a4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:08.554778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60b3e2-00e7-4018-814a-5f43bdb01a4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:08.556262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60b3e2-00e7-4018-814a-5f43bdb01a4a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:08.556594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 345/682] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 9 13:02:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:08.557938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fd08d-91ce-4629-a996-09438ef480dd None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:08.558389 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fd08d-91ce-4629-a996-09438ef480dd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:08.558663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fd08d-91ce-4629-a996-09438ef480dd None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:08.560497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 346/683] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:08.566008 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fd08d-91ce-4629-a996-09438ef480dd None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:08.571206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 338/684] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:08.572495 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fda2c-af3e-4005-a8a9-03ba9306db08 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:08.572772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fda2c-af3e-4005-a8a9-03ba9306db08 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:08.573040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fda2c-af3e-4005-a8a9-03ba9306db08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:08.580913 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:08.605409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-206fda2c-af3e-4005-a8a9-03ba9306db08 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:08.611024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-206fda2c-af3e-4005-a8a9-03ba9306db08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:08.697308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 339/685] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:02:08.698878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fda2c-af3e-4005-a8a9-03ba9306db08 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:08.712673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fd08d-91ce-4629-a996-09438ef480dd None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:08.782462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cace747-6dd8-4554-afaa-8449ec1db105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:08.784127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cace747-6dd8-4554-afaa-8449ec1db105 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:08.784401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cace747-6dd8-4554-afaa-8449ec1db105 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:08.784673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cace747-6dd8-4554-afaa-8449ec1db105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:08.848506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:08.890674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fda2c-af3e-4005-a8a9-03ba9306db08 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:08.915606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cace747-6dd8-4554-afaa-8449ec1db105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:08.917080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cace747-6dd8-4554-afaa-8449ec1db105 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:08.931361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:08.964404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e22b12-0e13-417f-94cd-dbc1530cbde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:08.964943 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e22b12-0e13-417f-94cd-dbc1530cbde6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:08.965215 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e22b12-0e13-417f-94cd-dbc1530cbde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:08.965482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e22b12-0e13-417f-94cd-dbc1530cbde6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:08.966205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e22b12-0e13-417f-94cd-dbc1530cbde6 None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=e12deabe077b4095bfa7e2e8fac61d14) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:08.966515 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e22b12-0e13-417f-94cd-dbc1530cbde6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:08.967989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e22b12-0e13-417f-94cd-dbc1530cbde6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:08.980088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 340/686] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:08 2022] GET /identity/v3/role_assignments?user.id=e12deabe077b4095bfa7e2e8fac61d14&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:09.025717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e22b12-0e13-417f-94cd-dbc1530cbde6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:09.102855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 347/687] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:02:09.107030 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cace747-6dd8-4554-afaa-8449ec1db105 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:09.238324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cace747-6dd8-4554-afaa-8449ec1db105 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:09.246675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:09.251616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e22b12-0e13-417f-94cd-dbc1530cbde6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:09.279985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1143e5-f5e9-41c2-91f2-7ad193872e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:09.281025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1143e5-f5e9-41c2-91f2-7ad193872e2d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:09.281523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1143e5-f5e9-41c2-91f2-7ad193872e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:09.281998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1143e5-f5e9-41c2-91f2-7ad193872e2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:09.294259 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1143e5-f5e9-41c2-91f2-7ad193872e2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:09.296502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1143e5-f5e9-41c2-91f2-7ad193872e2d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:09.302575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f55c1-e8cc-44f8-ac67-bace77a2e8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:09.303258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f55c1-e8cc-44f8-ac67-bace77a2e8de None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:09.303549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f55c1-e8cc-44f8-ac67-bace77a2e8de None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:09.303797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f55c1-e8cc-44f8-ac67-bace77a2e8de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7d1143e5-f5e9-41c2-91f2-7ad193872e2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:09.315355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:09.316327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:09.317241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:09.317241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:09.317241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:09.317422 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f55c1-e8cc-44f8-ac67-bace77a2e8de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:09.319417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 348/688] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-930f55c1-e8cc-44f8-ac67-bace77a2e8de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:09.327081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:09.328286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:09.329331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:09.329331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:09.329331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:09.329331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:09.329331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:09.329331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:09.329331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:09.329331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:09.329331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:09.329331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:09.329331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:09.329331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:09.329331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 341/689] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:02:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:09.330761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1143e5-f5e9-41c2-91f2-7ad193872e2d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:09.332634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f55c1-e8cc-44f8-ac67-bace77a2e8de None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:09.337731 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:09.456543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1143e5-f5e9-41c2-91f2-7ad193872e2d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:09.468516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f55c1-e8cc-44f8-ac67-bace77a2e8de None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:09.504914 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9723601-4d35-46b8-9287-eb5e9b4fec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:09.505767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9723601-4d35-46b8-9287-eb5e9b4fec00 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:09.506181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9723601-4d35-46b8-9287-eb5e9b4fec00 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:09.506598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9723601-4d35-46b8-9287-eb5e9b4fec00 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:09.507420 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9723601-4d35-46b8-9287-eb5e9b4fec00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:09.507914 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9723601-4d35-46b8-9287-eb5e9b4fec00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:09.509437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9723601-4d35-46b8-9287-eb5e9b4fec00 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:09.522667 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee6793-1d7e-417c-a607-9bbdbea8d25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:09.523590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee6793-1d7e-417c-a607-9bbdbea8d25b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:09.524043 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee6793-1d7e-417c-a607-9bbdbea8d25b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:09.524488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee6793-1d7e-417c-a607-9bbdbea8d25b None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:09.525342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee6793-1d7e-417c-a607-9bbdbea8d25b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:09.525818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee6793-1d7e-417c-a607-9bbdbea8d25b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:09.527370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee6793-1d7e-417c-a607-9bbdbea8d25b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:09.537523 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 342/691] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:02:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:09.537678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 349/691] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:09.542895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9723601-4d35-46b8-9287-eb5e9b4fec00 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:09.543502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee6793-1d7e-417c-a607-9bbdbea8d25b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:09.571789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:09.670212 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:09.714144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9723601-4d35-46b8-9287-eb5e9b4fec00 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:09.730673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee6793-1d7e-417c-a607-9bbdbea8d25b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:09.778857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f9879-82eb-44cb-9853-69293cba8b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:09.779434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f9879-82eb-44cb-9853-69293cba8b2f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:09.779713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f9879-82eb-44cb-9853-69293cba8b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:09.779981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f9879-82eb-44cb-9853-69293cba8b2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:09.786969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1225b4a-54b5-4e6f-84df-d2a4a69848e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:09.787655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1225b4a-54b5-4e6f-84df-d2a4a69848e2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:09.787934 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1225b4a-54b5-4e6f-84df-d2a4a69848e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:09.788206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1225b4a-54b5-4e6f-84df-d2a4a69848e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:09.795991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f9879-82eb-44cb-9853-69293cba8b2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:09.797455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f9879-82eb-44cb-9853-69293cba8b2f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:09.801891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1225b4a-54b5-4e6f-84df-d2a4a69848e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:09.803557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1225b4a-54b5-4e6f-84df-d2a4a69848e2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a1225b4a-54b5-4e6f-84df-d2a4a69848e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:09.818400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:09.819435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:09.820335 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:09.820335 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:09.820335 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:09.820335 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 343/692] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-996f9879-82eb-44cb-9853-69293cba8b2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:09.822598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:09.823560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:09.824466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:09.824466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:09.824466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:09.824466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:09.824466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:09.824466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:09.824466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:09.824466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:09.826774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1225b4a-54b5-4e6f-84df-d2a4a69848e2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:09.828354 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 350/693] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:09.830947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f9879-82eb-44cb-9853-69293cba8b2f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:09.978520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:09.986587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1225b4a-54b5-4e6f-84df-d2a4a69848e2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:10.014705 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:10.023330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f9879-82eb-44cb-9853-69293cba8b2f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:10.048877 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7afc3-6f3b-4c7f-8e25-698177ecc45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:10.049419 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7afc3-6f3b-4c7f-8e25-698177ecc45b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:10.049696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7afc3-6f3b-4c7f-8e25-698177ecc45b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:10.049968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7afc3-6f3b-4c7f-8e25-698177ecc45b None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:10.050613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7afc3-6f3b-4c7f-8e25-698177ecc45b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:10.050917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7afc3-6f3b-4c7f-8e25-698177ecc45b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:10.052316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7afc3-6f3b-4c7f-8e25-698177ecc45b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:10.080076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 344/694] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:10.081330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e04f1a-ad43-4e0c-88b3-9ec906dd6075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:10.081946 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e04f1a-ad43-4e0c-88b3-9ec906dd6075 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:10.082225 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e04f1a-ad43-4e0c-88b3-9ec906dd6075 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:10.082499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e04f1a-ad43-4e0c-88b3-9ec906dd6075 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:10.083230 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e04f1a-ad43-4e0c-88b3-9ec906dd6075 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:10.083539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e04f1a-ad43-4e0c-88b3-9ec906dd6075 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:10.085204 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7afc3-6f3b-4c7f-8e25-698177ecc45b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:10.088522 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e04f1a-ad43-4e0c-88b3-9ec906dd6075 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:10.112137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 351/695] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:10.119554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e04f1a-ad43-4e0c-88b3-9ec906dd6075 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:10.275736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7afc3-6f3b-4c7f-8e25-698177ecc45b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:10.311971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e04f1a-ad43-4e0c-88b3-9ec906dd6075 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:10.318607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd3908-e39c-4c1b-b7f7-dbe3605aaf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:10.319275 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd3908-e39c-4c1b-b7f7-dbe3605aaf4d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:10.319604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd3908-e39c-4c1b-b7f7-dbe3605aaf4d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:10.319881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd3908-e39c-4c1b-b7f7-dbe3605aaf4d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:10.327347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd3908-e39c-4c1b-b7f7-dbe3605aaf4d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:10.328923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd3908-e39c-4c1b-b7f7-dbe3605aaf4d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-badd3908-e39c-4c1b-b7f7-dbe3605aaf4d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:10.342199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:10.343235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:10.344213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:10.344213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:10.344213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:10.344213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:10.344213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:10.344213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:10.344213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:10.344213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:10.344213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:02:10.344213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:10.344213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 345/696] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 13:02:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:10.344804 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49582aba-29a0-47fd-beb7-987fe96d8e7f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:10.345907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49582aba-29a0-47fd-beb7-987fe96d8e7f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:10.345907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49582aba-29a0-47fd-beb7-987fe96d8e7f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:10.347419 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 346/697] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:10.351093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49582aba-29a0-47fd-beb7-987fe96d8e7f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:10.358498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2d6c1-23ed-483d-aed3-ee8f4901dd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:10.359146 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2d6c1-23ed-483d-aed3-ee8f4901dd88 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:10.359425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2d6c1-23ed-483d-aed3-ee8f4901dd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:10.359697 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2d6c1-23ed-483d-aed3-ee8f4901dd88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:10.360431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2d6c1-23ed-483d-aed3-ee8f4901dd88 None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=c361a3e83f8e45c2b4d28f97f0a9d069) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:10.361752 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2d6c1-23ed-483d-aed3-ee8f4901dd88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:10.363234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2d6c1-23ed-483d-aed3-ee8f4901dd88 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:10.370159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 352/698] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:10 2022] GET /identity/v3/role_assignments?user.id=c361a3e83f8e45c2b4d28f97f0a9d069&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:10.371598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d606d-ce6f-4f39-be2a-003576e0526c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:10.371989 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d606d-ce6f-4f39-be2a-003576e0526c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:10.372266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d606d-ce6f-4f39-be2a-003576e0526c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:10.403468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:10.408429 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8a0d606d-ce6f-4f39-be2a-003576e0526c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:10.409639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8a0d606d-ce6f-4f39-be2a-003576e0526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:10.421486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:10.552135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 353/699] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:02:10.561547 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49582aba-29a0-47fd-beb7-987fe96d8e7f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:10.619089 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94311a0e-bf2f-4c1c-977c-6310ed358a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:10.619756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94311a0e-bf2f-4c1c-977c-6310ed358a2e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:10.620036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94311a0e-bf2f-4c1c-977c-6310ed358a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:10.620308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94311a0e-bf2f-4c1c-977c-6310ed358a2e None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:10.621092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94311a0e-bf2f-4c1c-977c-6310ed358a2e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:10.621448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94311a0e-bf2f-4c1c-977c-6310ed358a2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:10.623021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94311a0e-bf2f-4c1c-977c-6310ed358a2e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:10.646296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 347/700] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 13:02:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:10.649388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d606d-ce6f-4f39-be2a-003576e0526c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:10.771090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d606d-ce6f-4f39-be2a-003576e0526c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:10.801823 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:10.807729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d6612-08c4-46e5-802c-8078a8e144c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:10.808350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d6612-08c4-46e5-802c-8078a8e144c6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:10.808626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d6612-08c4-46e5-802c-8078a8e144c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:10.808897 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d6612-08c4-46e5-802c-8078a8e144c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:10.839945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037382a-b50a-4adc-992a-f29c75632549 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:10.840483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037382a-b50a-4adc-992a-f29c75632549 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:10.840483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037382a-b50a-4adc-992a-f29c75632549 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:10.842070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 348/701] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:10.847517 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52659c86-0a1c-443e-bd1c-c22b7079db01 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:10.847907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52659c86-0a1c-443e-bd1c-c22b7079db01 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:10.848278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52659c86-0a1c-443e-bd1c-c22b7079db01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:10.871891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-52659c86-0a1c-443e-bd1c-c22b7079db01 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:10.873728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-52659c86-0a1c-443e-bd1c-c22b7079db01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:10.900738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d6612-08c4-46e5-802c-8078a8e144c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:10.903066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d6612-08c4-46e5-802c-8078a8e144c6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:10.965407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 349/702] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:02:10.966097 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52659c86-0a1c-443e-bd1c-c22b7079db01 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:10.997443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:11.086188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52659c86-0a1c-443e-bd1c-c22b7079db01 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:11.091629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 354/703] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:02:11.103983 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d6612-08c4-46e5-802c-8078a8e144c6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:11.138775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4154a-3c51-41e2-a7d3-62811a9864fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:11.139642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4154a-3c51-41e2-a7d3-62811a9864fb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:11.139923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4154a-3c51-41e2-a7d3-62811a9864fb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:11.140194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4154a-3c51-41e2-a7d3-62811a9864fb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:11.140783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4154a-3c51-41e2-a7d3-62811a9864fb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:11.147127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4154a-3c51-41e2-a7d3-62811a9864fb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:11.155653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-78e4154a-3c51-41e2-a7d3-62811a9864fb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:11.171596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:11.173099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 13:02:11.174259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:11.176634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 350/704] 10.222.0.17 () {56 vars in 1129 bytes} [Fri Sep 9 13:02:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:11.188293 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4154a-3c51-41e2-a7d3-62811a9864fb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:11.255302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d6612-08c4-46e5-802c-8078a8e144c6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:11.291471 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8fe78-fbf7-44e7-be9d-34333a50579d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:11.292494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8fe78-fbf7-44e7-be9d-34333a50579d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:11.292870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8fe78-fbf7-44e7-be9d-34333a50579d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:11.293219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8fe78-fbf7-44e7-be9d-34333a50579d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:11.304577 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8fe78-fbf7-44e7-be9d-34333a50579d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:11.306161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8fe78-fbf7-44e7-be9d-34333a50579d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:11.319468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-caf8fe78-fbf7-44e7-be9d-34333a50579d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:11.325023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:11.326002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:11.326886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:11.326886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:11.326886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:11.326886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 355/705] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:11.327794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8fe78-fbf7-44e7-be9d-34333a50579d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:11.336823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4154a-3c51-41e2-a7d3-62811a9864fb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:11.375480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24e8e0-5550-4ce0-a2d0-384d84f0c64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:11.376670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24e8e0-5550-4ce0-a2d0-384d84f0c64f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:11.377215 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24e8e0-5550-4ce0-a2d0-384d84f0c64f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:11.377678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24e8e0-5550-4ce0-a2d0-384d84f0c64f None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:11.378545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24e8e0-5550-4ce0-a2d0-384d84f0c64f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:11.379029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24e8e0-5550-4ce0-a2d0-384d84f0c64f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:11.380833 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24e8e0-5550-4ce0-a2d0-384d84f0c64f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:11.386425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 351/706] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 9 13:02:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:11.387613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24e8e0-5550-4ce0-a2d0-384d84f0c64f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:11.426842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:11.450880 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8fe78-fbf7-44e7-be9d-34333a50579d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:11.499040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4296aab-2590-4307-9f92-211bfe02d401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:11.500092 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4296aab-2590-4307-9f92-211bfe02d401 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:11.500591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4296aab-2590-4307-9f92-211bfe02d401 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:11.501094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4296aab-2590-4307-9f92-211bfe02d401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:11.509492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24e8e0-5550-4ce0-a2d0-384d84f0c64f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:11.512808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4296aab-2590-4307-9f92-211bfe02d401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:11.514365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4296aab-2590-4307-9f92-211bfe02d401 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d4296aab-2590-4307-9f92-211bfe02d401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:11.530314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:11.531500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:11.533237 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:11.533237 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:11.533237 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:11.538720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 356/707] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:11.539775 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4296aab-2590-4307-9f92-211bfe02d401 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:11.555993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4085ad-fb65-4339-89c8-311b79c84e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:11.557367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4085ad-fb65-4339-89c8-311b79c84e8d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:11.557799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4085ad-fb65-4339-89c8-311b79c84e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:11.558493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4085ad-fb65-4339-89c8-311b79c84e8d None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:11.559322 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4085ad-fb65-4339-89c8-311b79c84e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:11.560053 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4085ad-fb65-4339-89c8-311b79c84e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:11.561985 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4085ad-fb65-4339-89c8-311b79c84e8d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:11.583483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 352/708] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:11.585124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4085ad-fb65-4339-89c8-311b79c84e8d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:11.605886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:11.661129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4296aab-2590-4307-9f92-211bfe02d401 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:11.702783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bed47c-bba5-48d6-a5c8-8fe170f2ff96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:11.703537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bed47c-bba5-48d6-a5c8-8fe170f2ff96 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:11.704194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bed47c-bba5-48d6-a5c8-8fe170f2ff96 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:11.704368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bed47c-bba5-48d6-a5c8-8fe170f2ff96 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:11.705456 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bed47c-bba5-48d6-a5c8-8fe170f2ff96 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:11.705841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bed47c-bba5-48d6-a5c8-8fe170f2ff96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:11.707345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bed47c-bba5-48d6-a5c8-8fe170f2ff96 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:11.711573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:11.719528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 357/709] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 9 13:02:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:11.721274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bed47c-bba5-48d6-a5c8-8fe170f2ff96 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:11.734460 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4085ad-fb65-4339-89c8-311b79c84e8d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:11.781536 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37eee6-c3f5-461a-b77c-aef0405488df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:11.783025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37eee6-c3f5-461a-b77c-aef0405488df None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:11.783524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37eee6-c3f5-461a-b77c-aef0405488df None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:11.784347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37eee6-c3f5-461a-b77c-aef0405488df None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:11.785154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37eee6-c3f5-461a-b77c-aef0405488df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:11.785607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37eee6-c3f5-461a-b77c-aef0405488df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:11.787238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37eee6-c3f5-461a-b77c-aef0405488df None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:11.802314 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 353/710] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:11.809843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37eee6-c3f5-461a-b77c-aef0405488df None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:11.918703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bed47c-bba5-48d6-a5c8-8fe170f2ff96 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:11.935495 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:11.964519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c1294-b017-456a-8f2f-d8b4d3736c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:11.965500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c1294-b017-456a-8f2f-d8b4d3736c3a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:11.965994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c1294-b017-456a-8f2f-d8b4d3736c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:11.966473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c1294-b017-456a-8f2f-d8b4d3736c3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:11.973856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37eee6-c3f5-461a-b77c-aef0405488df None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:11.977281 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c1294-b017-456a-8f2f-d8b4d3736c3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:11.978299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c1294-b017-456a-8f2f-d8b4d3736c3a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e23c1294-b017-456a-8f2f-d8b4d3736c3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:11.994125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:11.995114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:11.996050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:11.996050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:11.996050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:11.996050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:11.996050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:11.996050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:11.996050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:11.996050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:11.996050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 358/711] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:11.997017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f88a24-8786-46e1-9dc4-12e79901b6e5 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:11.997017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f88a24-8786-46e1-9dc4-12e79901b6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:11.997174 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f88a24-8786-46e1-9dc4-12e79901b6e5 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:11.998648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 359/712] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:12.007214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f88a24-8786-46e1-9dc4-12e79901b6e5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:12.019929 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d34c61-8b73-4ffd-b726-b55e92b087e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:12.020592 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d34c61-8b73-4ffd-b726-b55e92b087e3 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:12.021244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d34c61-8b73-4ffd-b726-b55e92b087e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:12.021526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d34c61-8b73-4ffd-b726-b55e92b087e3 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:12.022471 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d34c61-8b73-4ffd-b726-b55e92b087e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:12.024252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d34c61-8b73-4ffd-b726-b55e92b087e3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:12.052837 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 354/713] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:02:11 2022] POST /identity/v3/users => generated 254 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:02:12.054095 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fbd14-60da-4f5e-bf9b-a339631c4df0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:12.054382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fbd14-60da-4f5e-bf9b-a339631c4df0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:12.054659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fbd14-60da-4f5e-bf9b-a339631c4df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:12.086399 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2c1fbd14-60da-4f5e-bf9b-a339631c4df0 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:12.091098 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2c1fbd14-60da-4f5e-bf9b-a339631c4df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:12.121679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:12.166732 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 355/714] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:02:12.207306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f88a24-8786-46e1-9dc4-12e79901b6e5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:12.257834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203f3f2-7b44-4212-8a76-56aed7787815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:12.258757 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203f3f2-7b44-4212-8a76-56aed7787815 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:12.259241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203f3f2-7b44-4212-8a76-56aed7787815 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:12.259718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203f3f2-7b44-4212-8a76-56aed7787815 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:12.260612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203f3f2-7b44-4212-8a76-56aed7787815 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:12.261141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203f3f2-7b44-4212-8a76-56aed7787815 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:12.262818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203f3f2-7b44-4212-8a76-56aed7787815 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:12.278643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 360/715] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:12.285361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fbd14-60da-4f5e-bf9b-a339631c4df0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:12.353419 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:12.404181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fbd14-60da-4f5e-bf9b-a339631c4df0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:12.442757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39d0e0-e10f-42e1-874e-1e54ea42f131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:12.443734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39d0e0-e10f-42e1-874e-1e54ea42f131 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:12.444199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39d0e0-e10f-42e1-874e-1e54ea42f131 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:12.444713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39d0e0-e10f-42e1-874e-1e54ea42f131 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:12.445659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39d0e0-e10f-42e1-874e-1e54ea42f131 None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=441575b8b496457b905e9d5f5846e092) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:12.446153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39d0e0-e10f-42e1-874e-1e54ea42f131 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:12.447799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39d0e0-e10f-42e1-874e-1e54ea42f131 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:12.458019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 356/716] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:12 2022] GET /identity/v3/role_assignments?user.id=441575b8b496457b905e9d5f5846e092&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:12.494979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203f3f2-7b44-4212-8a76-56aed7787815 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:12.639235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203f3f2-7b44-4212-8a76-56aed7787815 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:12.673248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:12.700629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523ddc9-03d5-4ed7-b966-bd672c88d9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:12.701675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523ddc9-03d5-4ed7-b966-bd672c88d9c8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:12.702142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523ddc9-03d5-4ed7-b966-bd672c88d9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:12.702596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523ddc9-03d5-4ed7-b966-bd672c88d9c8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:12.703403 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523ddc9-03d5-4ed7-b966-bd672c88d9c8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:12.705158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523ddc9-03d5-4ed7-b966-bd672c88d9c8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6523ddc9-03d5-4ed7-b966-bd672c88d9c8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:12.718834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:12.720120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 13:02:12.720999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:12.723443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 361/717] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:12 2022] GET /identity/v3/services/network => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:12.728924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39d0e0-e10f-42e1-874e-1e54ea42f131 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:12.849400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39d0e0-e10f-42e1-874e-1e54ea42f131 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:12.878084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:12.890621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2189cf9-d441-4ee8-91f0-3c600616dac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:12.892022 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2189cf9-d441-4ee8-91f0-3c600616dac0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:12.892479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2189cf9-d441-4ee8-91f0-3c600616dac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:12.892939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2189cf9-d441-4ee8-91f0-3c600616dac0 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:12.893715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2189cf9-d441-4ee8-91f0-3c600616dac0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:12.894164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2189cf9-d441-4ee8-91f0-3c600616dac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:12.895818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2189cf9-d441-4ee8-91f0-3c600616dac0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:12.900995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 357/718] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 13:02:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:12.909129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523ddc9-03d5-4ed7-b966-bd672c88d9c8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:13.040314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523ddc9-03d5-4ed7-b966-bd672c88d9c8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:13.090735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1d91b-5534-41c2-a96f-b9f479eec9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:13.091628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1d91b-5534-41c2-a96f-b9f479eec9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:13.092085 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1d91b-5534-41c2-a96f-b9f479eec9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:13.092624 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1d91b-5534-41c2-a96f-b9f479eec9b5 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:13.093469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1d91b-5534-41c2-a96f-b9f479eec9b5 None admin] RBAC: Adding query filter params (type=network) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:13.093952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1d91b-5534-41c2-a96f-b9f479eec9b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:13.096335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1d91b-5534-41c2-a96f-b9f479eec9b5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:13.107121 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 362/719] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 13:02:12 2022] GET /identity/v3/services?type=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:13.179906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199e28f-970a-4756-88a9-d5663761ac32 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:13.180769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199e28f-970a-4756-88a9-d5663761ac32 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:13.181197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199e28f-970a-4756-88a9-d5663761ac32 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:13.182781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 358/720] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:13.189452 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88799a-2460-47a4-948c-f38030639dd7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:13.189988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88799a-2460-47a4-948c-f38030639dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:13.190428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88799a-2460-47a4-948c-f38030639dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:13.220172 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9f88799a-2460-47a4-948c-f38030639dd7 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:13.221903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9f88799a-2460-47a4-948c-f38030639dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:13.285599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:13.321322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 363/721] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:02:13.636572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199e28f-970a-4756-88a9-d5663761ac32 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:13.744918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199e28f-970a-4756-88a9-d5663761ac32 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:13.781380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688f65f-f811-41ef-b3c4-736845f15f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:13.782454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688f65f-f811-41ef-b3c4-736845f15f31 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:13.782884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688f65f-f811-41ef-b3c4-736845f15f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:13.783346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688f65f-f811-41ef-b3c4-736845f15f31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:13.796004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688f65f-f811-41ef-b3c4-736845f15f31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f688f65f-f811-41ef-b3c4-736845f15f31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:13.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:13.803553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:13.804670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:13.804670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:13.804670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:13.804670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:13.804670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:13.804670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:13.804670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:13.804670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:13.804670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:13.804670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:13.804670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:13.804670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:13.807009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 359/722] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:02:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:13.816162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88799a-2460-47a4-948c-f38030639dd7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:13.960501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88799a-2460-47a4-948c-f38030639dd7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:14.007916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e569934-c48b-4580-92cf-2638bee5905d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:14.008593 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e569934-c48b-4580-92cf-2638bee5905d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:14.008881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e569934-c48b-4580-92cf-2638bee5905d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:14.009155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e569934-c48b-4580-92cf-2638bee5905d None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:14.009821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e569934-c48b-4580-92cf-2638bee5905d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:14.010123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e569934-c48b-4580-92cf-2638bee5905d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:14.011510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e569934-c48b-4580-92cf-2638bee5905d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:14.020600 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 364/723] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:02:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:14.026064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688f65f-f811-41ef-b3c4-736845f15f31 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:14.104604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:14.165475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688f65f-f811-41ef-b3c4-736845f15f31 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:14.217116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078253b0-64a1-4773-a668-b3e62574d17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:14.217756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078253b0-64a1-4773-a668-b3e62574d17b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:14.217991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078253b0-64a1-4773-a668-b3e62574d17b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:14.218250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078253b0-64a1-4773-a668-b3e62574d17b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:14.230922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078253b0-64a1-4773-a668-b3e62574d17b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:14.232444 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078253b0-64a1-4773-a668-b3e62574d17b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-078253b0-64a1-4773-a668-b3e62574d17b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:14.254577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:14.255634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:14.256621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:14.256621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:14.256621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:14.260140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 360/724] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:14.265759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e569934-c48b-4580-92cf-2638bee5905d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:14.335222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:14.419086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e569934-c48b-4580-92cf-2638bee5905d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:14.448651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ce55-b490-44a5-b226-8d23aa51e8bf None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:14.449271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ce55-b490-44a5-b226-8d23aa51e8bf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:14.449748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ce55-b490-44a5-b226-8d23aa51e8bf None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:14.452790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 361/725] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:02:14.457716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f3b17-4f0c-4589-b0c2-4baff7d1f6ba None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:14.458003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f3b17-4f0c-4589-b0c2-4baff7d1f6ba None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:14.458276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f3b17-4f0c-4589-b0c2-4baff7d1f6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:14.467730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bd85c-a6ce-411a-be6f-df7ac50ed4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:14.468284 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bd85c-a6ce-411a-be6f-df7ac50ed4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:14.468564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bd85c-a6ce-411a-be6f-df7ac50ed4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:14.468834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bd85c-a6ce-411a-be6f-df7ac50ed4b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:14.470039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bd85c-a6ce-411a-be6f-df7ac50ed4b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:14.470458 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bd85c-a6ce-411a-be6f-df7ac50ed4b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:14.471842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bd85c-a6ce-411a-be6f-df7ac50ed4b8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:14.494021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-974f3b17-4f0c-4589-b0c2-4baff7d1f6ba None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:14.495290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-974f3b17-4f0c-4589-b0c2-4baff7d1f6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:14.499989 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 365/726] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:14.505801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bd85c-a6ce-411a-be6f-df7ac50ed4b8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:14.607488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 362/727] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:02:14.662592 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:14.665975 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bd85c-a6ce-411a-be6f-df7ac50ed4b8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:14.705247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e889b-3cc6-4de7-b638-c53fef5f16ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:14.705247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e889b-3cc6-4de7-b638-c53fef5f16ad None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:14.705247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e889b-3cc6-4de7-b638-c53fef5f16ad None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:14.705247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e889b-3cc6-4de7-b638-c53fef5f16ad None admin] PATH_INFO: `/v3/users/swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:14.712377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e889b-3cc6-4de7-b638-c53fef5f16ad None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:14.713926 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e889b-3cc6-4de7-b638-c53fef5f16ad None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-1b0e889b-3cc6-4de7-b638-c53fef5f16ad None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:14.726109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:14.727127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:14.728262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:14.728262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:14.728262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:14.728262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:14.728262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:14.728262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:14.728262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:14.728262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:14.728262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 13:02:14.728262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:14.728262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 366/728] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:02:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:14.739200 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f3b17-4f0c-4589-b0c2-4baff7d1f6ba None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:14.784333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:14.884952 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f3b17-4f0c-4589-b0c2-4baff7d1f6ba None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:14.932864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aca746-d3b0-4b84-a65d-2a25584b27da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:14.933492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca746-d3b0-4b84-a65d-2a25584b27da None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:14.933765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca746-d3b0-4b84-a65d-2a25584b27da None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:14.934032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca746-d3b0-4b84-a65d-2a25584b27da None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:14.934759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aca746-d3b0-4b84-a65d-2a25584b27da None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:14.935082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aca746-d3b0-4b84-a65d-2a25584b27da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:14.936522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aca746-d3b0-4b84-a65d-2a25584b27da None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:14.939949 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e889b-3cc6-4de7-b638-c53fef5f16ad None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:14.973276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 363/729] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 9 13:02:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:14.976841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aca746-d3b0-4b84-a65d-2a25584b27da None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:15.135425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:15.137688 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e889b-3cc6-4de7-b638-c53fef5f16ad None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:15.149144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aca746-d3b0-4b84-a65d-2a25584b27da None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:15.176200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d380f85-36ec-451d-8368-ddddf0a038fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:15.176925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d380f85-36ec-451d-8368-ddddf0a038fa None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:15.177209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d380f85-36ec-451d-8368-ddddf0a038fa None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:15.177479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d380f85-36ec-451d-8368-ddddf0a038fa None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:15.178126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d380f85-36ec-451d-8368-ddddf0a038fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:15.179659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d380f85-36ec-451d-8368-ddddf0a038fa None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:15.193475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa34ad0-0ac8-496e-8057-7a7de9502cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:15.194360 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa34ad0-0ac8-496e-8057-7a7de9502cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:15.194639 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa34ad0-0ac8-496e-8057-7a7de9502cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:15.194911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa34ad0-0ac8-496e-8057-7a7de9502cc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:15.200343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 367/730] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:02:14 2022] POST /identity/v3/services => generated 251 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:02:15.201813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd46eee-e3fa-435a-9838-7208c0eb3dd2 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:15.202215 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd46eee-e3fa-435a-9838-7208c0eb3dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:15.202545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd46eee-e3fa-435a-9838-7208c0eb3dd2 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:15.205077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 368/731] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:15.221076 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06622bf-47d2-4f97-9058-cc5ba51c8993 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:15.221389 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06622bf-47d2-4f97-9058-cc5ba51c8993 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:15.221636 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06622bf-47d2-4f97-9058-cc5ba51c8993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:15.280315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e06622bf-47d2-4f97-9058-cc5ba51c8993 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:15.281021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa34ad0-0ac8-496e-8057-7a7de9502cc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:15.282502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e06622bf-47d2-4f97-9058-cc5ba51c8993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:15.282962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa34ad0-0ac8-496e-8057-7a7de9502cc8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:15.330859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:15.406436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:15.423732 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 364/732] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 13:02:15.430825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa34ad0-0ac8-496e-8057-7a7de9502cc8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:15.492244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 369/733] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:02:15.493641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca7e17-0636-446a-92b6-9310ad5c5350 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:15.493991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca7e17-0636-446a-92b6-9310ad5c5350 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:15.494267 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca7e17-0636-446a-92b6-9310ad5c5350 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:15.495970 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 370/734] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:15.506451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ad2ef-3f5f-4042-931d-6e63099dbc55 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:15.506875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ad2ef-3f5f-4042-931d-6e63099dbc55 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:15.507210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ad2ef-3f5f-4042-931d-6e63099dbc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:15.537428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-488ad2ef-3f5f-4042-931d-6e63099dbc55 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:15.539793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-488ad2ef-3f5f-4042-931d-6e63099dbc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:15.555667 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa34ad0-0ac8-496e-8057-7a7de9502cc8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:15.610415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0c673-f88f-4161-9335-dad9cabe9b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:15.611585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c673-f88f-4161-9335-dad9cabe9b52 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:15.612068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c673-f88f-4161-9335-dad9cabe9b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:15.612561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c673-f88f-4161-9335-dad9cabe9b52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:15.626185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 371/735] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 13:02:15.629713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0c673-f88f-4161-9335-dad9cabe9b52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:15.631456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0c673-f88f-4161-9335-dad9cabe9b52 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4ba0c673-f88f-4161-9335-dad9cabe9b52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:15.647924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:15.648913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:15.649799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:15.649799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:15.649799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:15.651993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 365/736] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:15.658460 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ad2ef-3f5f-4042-931d-6e63099dbc55 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:15.763983 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:15.789343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ad2ef-3f5f-4042-931d-6e63099dbc55 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:15.828029 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a86bd-d564-44c9-9c5f-380edb1bdf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:15.828666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a86bd-d564-44c9-9c5f-380edb1bdf93 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:15.828944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a86bd-d564-44c9-9c5f-380edb1bdf93 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:15.829214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a86bd-d564-44c9-9c5f-380edb1bdf93 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:15.829860 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a86bd-d564-44c9-9c5f-380edb1bdf93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:15.830159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a86bd-d564-44c9-9c5f-380edb1bdf93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:15.831566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a86bd-d564-44c9-9c5f-380edb1bdf93 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:15.848404 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 372/737] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:15.850209 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0c673-f88f-4161-9335-dad9cabe9b52 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:15.919392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:15.931511 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a86bd-d564-44c9-9c5f-380edb1bdf93 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:15.997074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0c673-f88f-4161-9335-dad9cabe9b52 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:16.037065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c0f59-f058-40b4-9e5a-42353976576b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.038176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c0f59-f058-40b4-9e5a-42353976576b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.038665 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c0f59-f058-40b4-9e5a-42353976576b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.039171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c0f59-f058-40b4-9e5a-42353976576b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.046560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c0f59-f058-40b4-9e5a-42353976576b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:16.048989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c0f59-f058-40b4-9e5a-42353976576b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4d7c0f59-f058-40b4-9e5a-42353976576b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:16.061123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:16.062320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:16.063995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:16.063995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:16.063995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:16.063995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:16.063995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:16.063995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:16.063995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:16.063995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:16.066405 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 366/738] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:16.067504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c0f59-f058-40b4-9e5a-42353976576b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:16.098320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a86bd-d564-44c9-9c5f-380edb1bdf93 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:16.143628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc090e-1989-4b10-ae52-e36210e3df9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.144263 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc090e-1989-4b10-ae52-e36210e3df9c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.144548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc090e-1989-4b10-ae52-e36210e3df9c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.144818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc090e-1989-4b10-ae52-e36210e3df9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.148268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:16.157377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc090e-1989-4b10-ae52-e36210e3df9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-21cc090e-1989-4b10-ae52-e36210e3df9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:16.162833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:16.164169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:16.165597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:16.165597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:16.165597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:16.165597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:16.165597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:16.165597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:16.165597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:16.165597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:16.165597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:16.165597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:16.165597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:16.165597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:16.165597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 373/739] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:02:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:16.166306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc090e-1989-4b10-ae52-e36210e3df9c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:16.203257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:16.207325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c0f59-f058-40b4-9e5a-42353976576b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:16.247474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f63f5-00cc-483e-ae08-623cdc8d208f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.248101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f63f5-00cc-483e-ae08-623cdc8d208f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.248376 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f63f5-00cc-483e-ae08-623cdc8d208f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.248642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f63f5-00cc-483e-ae08-623cdc8d208f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.260414 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f63f5-00cc-483e-ae08-623cdc8d208f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bb4f63f5-00cc-483e-ae08-623cdc8d208f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:16.265675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:16.266674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:16.267572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:16.267572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:16.267572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:16.267572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:16.267572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:16.267572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:16.267572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:16.267572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:16.267572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:16.267572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:16.267572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:16.267572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:16.267572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 367/740] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:02:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:16.268810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66449130-8003-4c62-9c40-5bbcbdfb3db1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.268810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66449130-8003-4c62-9c40-5bbcbdfb3db1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.268968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66449130-8003-4c62-9c40-5bbcbdfb3db1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.270346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 368/741] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:16.271948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66449130-8003-4c62-9c40-5bbcbdfb3db1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:16.305353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc090e-1989-4b10-ae52-e36210e3df9c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:16.346165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf8066-53be-46c1-9f08-4975e5a025ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.347081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf8066-53be-46c1-9f08-4975e5a025ee None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.347639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf8066-53be-46c1-9f08-4975e5a025ee None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.348114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf8066-53be-46c1-9f08-4975e5a025ee None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.348988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf8066-53be-46c1-9f08-4975e5a025ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:16.349294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf8066-53be-46c1-9f08-4975e5a025ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:16.350735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf8066-53be-46c1-9f08-4975e5a025ee None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:16.364875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 374/742] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:16.366133 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89d2b2-acaf-4902-abad-102733bc689f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.366417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89d2b2-acaf-4902-abad-102733bc689f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.366689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89d2b2-acaf-4902-abad-102733bc689f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.386287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-da89d2b2-acaf-4902-abad-102733bc689f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:16.387584 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-da89d2b2-acaf-4902-abad-102733bc689f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:16.418445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66449130-8003-4c62-9c40-5bbcbdfb3db1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:16.444910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:16.460880 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c126bd1-2d17-4748-b6ce-1c1746e6ca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.461496 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c126bd1-2d17-4748-b6ce-1c1746e6ca76 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.461771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c126bd1-2d17-4748-b6ce-1c1746e6ca76 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.462044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c126bd1-2d17-4748-b6ce-1c1746e6ca76 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.462702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c126bd1-2d17-4748-b6ce-1c1746e6ca76 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:16.463005 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c126bd1-2d17-4748-b6ce-1c1746e6ca76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:16.464409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c126bd1-2d17-4748-b6ce-1c1746e6ca76 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:16.465881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 375/743] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:16 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 13:02:16.466739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89d2b2-acaf-4902-abad-102733bc689f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:16.480092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 369/744] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:02:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:16.485359 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c126bd1-2d17-4748-b6ce-1c1746e6ca76 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:16.514171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:16.584086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89d2b2-acaf-4902-abad-102733bc689f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:16.598125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c126bd1-2d17-4748-b6ce-1c1746e6ca76 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:16.621092 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a76a4-26a4-4217-a308-2547734f296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.621852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a76a4-26a4-4217-a308-2547734f296f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.622228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a76a4-26a4-4217-a308-2547734f296f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.622589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a76a4-26a4-4217-a308-2547734f296f None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.623395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a76a4-26a4-4217-a308-2547734f296f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:16.623816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a76a4-26a4-4217-a308-2547734f296f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:16.625496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a76a4-26a4-4217-a308-2547734f296f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:16.638649 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 376/745] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:02:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:16.638649 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a76a4-26a4-4217-a308-2547734f296f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:16.646076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c44aa-47fa-44db-9ab4-527658ab6e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.646890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c44aa-47fa-44db-9ab4-527658ab6e9f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.647345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c44aa-47fa-44db-9ab4-527658ab6e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.647758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c44aa-47fa-44db-9ab4-527658ab6e9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.648625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c44aa-47fa-44db-9ab4-527658ab6e9f None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=c361a3e83f8e45c2b4d28f97f0a9d069) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:16.649095 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c44aa-47fa-44db-9ab4-527658ab6e9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:16.650669 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c44aa-47fa-44db-9ab4-527658ab6e9f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:16.663494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 370/746] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:16 2022] GET /identity/v3/role_assignments?user.id=c361a3e83f8e45c2b4d28f97f0a9d069&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:16.664689 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c44aa-47fa-44db-9ab4-527658ab6e9f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:16.699082 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:16.754735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a76a4-26a4-4217-a308-2547734f296f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:16.793886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3b248-a254-4a04-94a8-b1e2e3b90090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.794570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b248-a254-4a04-94a8-b1e2e3b90090 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.794989 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b248-a254-4a04-94a8-b1e2e3b90090 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.795302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b248-a254-4a04-94a8-b1e2e3b90090 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.807660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3b248-a254-4a04-94a8-b1e2e3b90090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:16.809148 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3b248-a254-4a04-94a8-b1e2e3b90090 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:16.813379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8bb3b248-a254-4a04-94a8-b1e2e3b90090 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:16.825498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:16.826534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:16.827463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:16.827463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:16.827463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:16.827463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 377/747] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:16.828224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3b248-a254-4a04-94a8-b1e2e3b90090 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:16.841658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c44aa-47fa-44db-9ab4-527658ab6e9f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:16.889541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354931e-6fdc-40ac-8e12-2a00b51fc192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.890202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354931e-6fdc-40ac-8e12-2a00b51fc192 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.890481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354931e-6fdc-40ac-8e12-2a00b51fc192 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.890754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354931e-6fdc-40ac-8e12-2a00b51fc192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.908782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354931e-6fdc-40ac-8e12-2a00b51fc192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:16.910734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354931e-6fdc-40ac-8e12-2a00b51fc192 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5354931e-6fdc-40ac-8e12-2a00b51fc192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:16.933744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:16.934760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:16.935712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:16.935712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:16.935712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:16.935886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 371/748] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:16.941590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354931e-6fdc-40ac-8e12-2a00b51fc192 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:16.958034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:16.962174 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3b248-a254-4a04-94a8-b1e2e3b90090 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:17.013645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd01c6a-99a9-44ce-8cc5-0b4a1c37d8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:17.014646 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd01c6a-99a9-44ce-8cc5-0b4a1c37d8de None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:17.015021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd01c6a-99a9-44ce-8cc5-0b4a1c37d8de None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:17.015415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd01c6a-99a9-44ce-8cc5-0b4a1c37d8de None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:17.016331 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd01c6a-99a9-44ce-8cc5-0b4a1c37d8de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:17.018743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd01c6a-99a9-44ce-8cc5-0b4a1c37d8de None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:17.039399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 378/749] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:02:16 2022] POST /identity/v3/services => generated 240 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:17.040443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd01c6a-99a9-44ce-8cc5-0b4a1c37d8de None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:17.056997 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354931e-6fdc-40ac-8e12-2a00b51fc192 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:17.113584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a02c6-984e-48c6-8727-af43e27d632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:17.114558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a02c6-984e-48c6-8727-af43e27d632a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:17.115008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a02c6-984e-48c6-8727-af43e27d632a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:17.115439 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a02c6-984e-48c6-8727-af43e27d632a None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:17.116112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a02c6-984e-48c6-8727-af43e27d632a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:17.116421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a02c6-984e-48c6-8727-af43e27d632a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:17.118706 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a02c6-984e-48c6-8727-af43e27d632a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:17.120581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:17.143140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 372/750] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:17.149826 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a02c6-984e-48c6-8727-af43e27d632a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:17.198925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd01c6a-99a9-44ce-8cc5-0b4a1c37d8de None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:17.275099 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf30ed7-a321-428d-a67f-30de9c72e69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:17.275726 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf30ed7-a321-428d-a67f-30de9c72e69c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:17.276001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf30ed7-a321-428d-a67f-30de9c72e69c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:17.276271 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf30ed7-a321-428d-a67f-30de9c72e69c None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:17.277038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf30ed7-a321-428d-a67f-30de9c72e69c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:17.277337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf30ed7-a321-428d-a67f-30de9c72e69c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:17.278848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf30ed7-a321-428d-a67f-30de9c72e69c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:17.294417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 379/751] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:17.301881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf30ed7-a321-428d-a67f-30de9c72e69c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:17.318965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a02c6-984e-48c6-8727-af43e27d632a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:17.332476 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:17.364580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32d2d4-2c2b-4864-87f7-dc260a5c06b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:17.365189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32d2d4-2c2b-4864-87f7-dc260a5c06b4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:17.365508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32d2d4-2c2b-4864-87f7-dc260a5c06b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:17.365746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32d2d4-2c2b-4864-87f7-dc260a5c06b4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:17.374286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa32d2d4-2c2b-4864-87f7-dc260a5c06b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:17.376464 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa32d2d4-2c2b-4864-87f7-dc260a5c06b4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-aa32d2d4-2c2b-4864-87f7-dc260a5c06b4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:17.391425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:17.392852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:17.394070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:17.394070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:17.394070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:17.394070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:17.394070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:17.394070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:17.394070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:17.394070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:17.394070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 13:02:17.394070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:17.394070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 373/752] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 13:02:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:17.399743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32d2d4-2c2b-4864-87f7-dc260a5c06b4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:17.439912 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf30ed7-a321-428d-a67f-30de9c72e69c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:17.505284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:17.514460 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d828d-4263-4e7f-b370-339e6a50b7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:17.515136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d828d-4263-4e7f-b370-339e6a50b7db None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:17.515420 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d828d-4263-4e7f-b370-339e6a50b7db None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:17.516210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d828d-4263-4e7f-b370-339e6a50b7db None admin] PATH_INFO: `/v3/users/nova` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:17.525438 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d828d-4263-4e7f-b370-339e6a50b7db None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:17.526897 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d828d-4263-4e7f-b370-339e6a50b7db None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8a9d828d-4263-4e7f-b370-339e6a50b7db None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:17.539334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:17.540505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:17.541715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:17.541715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:17.541715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:17.541715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:17.541715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:17.541715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:17.541715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:17.541715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:17.541715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:02:17.541715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:17.541715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 380/753] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 13:02:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:17.548501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d828d-4263-4e7f-b370-339e6a50b7db None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:17.596494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32d2d4-2c2b-4864-87f7-dc260a5c06b4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:17.673801 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25eb0d-9338-484e-ac53-55057aa7893e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:17.674432 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25eb0d-9338-484e-ac53-55057aa7893e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:17.674716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25eb0d-9338-484e-ac53-55057aa7893e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:17.674995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25eb0d-9338-484e-ac53-55057aa7893e None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:17.675770 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25eb0d-9338-484e-ac53-55057aa7893e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:17.676081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25eb0d-9338-484e-ac53-55057aa7893e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:17.677526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25eb0d-9338-484e-ac53-55057aa7893e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:17.711841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:17.721242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 374/754] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 9 13:02:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:17.729620 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25eb0d-9338-484e-ac53-55057aa7893e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:17.752538 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d828d-4263-4e7f-b370-339e6a50b7db None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:17.822884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e370d-aa7c-44e5-8f80-f090dd118049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:17.823550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e370d-aa7c-44e5-8f80-f090dd118049 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:17.823833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e370d-aa7c-44e5-8f80-f090dd118049 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:17.824109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e370d-aa7c-44e5-8f80-f090dd118049 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:17.824862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e370d-aa7c-44e5-8f80-f090dd118049 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:17.825172 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e370d-aa7c-44e5-8f80-f090dd118049 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:17.826653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e370d-aa7c-44e5-8f80-f090dd118049 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:17.865735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 381/755] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 13:02:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:17.869479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e370d-aa7c-44e5-8f80-f090dd118049 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:17.912095 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25eb0d-9338-484e-ac53-55057aa7893e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:17.939163 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:17.971486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04935888-3caa-446c-8ce3-639b6f8289d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:17.972104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04935888-3caa-446c-8ce3-639b6f8289d1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:17.972382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04935888-3caa-446c-8ce3-639b6f8289d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:17.972648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04935888-3caa-446c-8ce3-639b6f8289d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:18.014014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e370d-aa7c-44e5-8f80-f090dd118049 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:18.054698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e22db9-cd2f-4dd5-bf29-1d4fc6148175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:18.055483 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e22db9-cd2f-4dd5-bf29-1d4fc6148175 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:18.055853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e22db9-cd2f-4dd5-bf29-1d4fc6148175 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:18.056184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e22db9-cd2f-4dd5-bf29-1d4fc6148175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:18.065204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:18.066610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04935888-3caa-446c-8ce3-639b6f8289d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:18.069258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e22db9-cd2f-4dd5-bf29-1d4fc6148175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:18.070037 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04935888-3caa-446c-8ce3-639b6f8289d1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:18.071270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e22db9-cd2f-4dd5-bf29-1d4fc6148175 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-28e22db9-cd2f-4dd5-bf29-1d4fc6148175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:18.090930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:18.092160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:18.093326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:18.093326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:18.093326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:18.094974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 382/756] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:18.096278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406acd93-4215-4a40-977c-c4d0094fa571 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:18.096812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406acd93-4215-4a40-977c-c4d0094fa571 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:18.096991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406acd93-4215-4a40-977c-c4d0094fa571 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:18.098576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 383/757] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:18.099938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406acd93-4215-4a40-977c-c4d0094fa571 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:18.200292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 375/758] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:02:18.202074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae228cf-ccbb-47cf-afa4-987d98592dfb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:18.202527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae228cf-ccbb-47cf-afa4-987d98592dfb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:18.202953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae228cf-ccbb-47cf-afa4-987d98592dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:18.218699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406acd93-4215-4a40-977c-c4d0094fa571 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:18.224604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0ae228cf-ccbb-47cf-afa4-987d98592dfb None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:18.226095 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0ae228cf-ccbb-47cf-afa4-987d98592dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:18.234983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:18.256908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bb122-51b8-4be8-9b9b-f49832cd35d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:18.257488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bb122-51b8-4be8-9b9b-f49832cd35d8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:18.257808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bb122-51b8-4be8-9b9b-f49832cd35d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:18.258380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bb122-51b8-4be8-9b9b-f49832cd35d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:18.258872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bb122-51b8-4be8-9b9b-f49832cd35d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:18.259211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bb122-51b8-4be8-9b9b-f49832cd35d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:18.260646 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bb122-51b8-4be8-9b9b-f49832cd35d8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:18.274477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 384/759] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:18.275859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bb122-51b8-4be8-9b9b-f49832cd35d8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:18.303081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 376/760] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:18.304246 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae228cf-ccbb-47cf-afa4-987d98592dfb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:18.376686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:18.410155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae228cf-ccbb-47cf-afa4-987d98592dfb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:18.446765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef237d-f914-4bd2-95ef-4394e743e805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:18.448583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef237d-f914-4bd2-95ef-4394e743e805 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:18.449016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef237d-f914-4bd2-95ef-4394e743e805 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:18.449626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef237d-f914-4bd2-95ef-4394e743e805 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:18.463462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef237d-f914-4bd2-95ef-4394e743e805 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:18.465309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef237d-f914-4bd2-95ef-4394e743e805 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-eaef237d-f914-4bd2-95ef-4394e743e805 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:18.483454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:18.484614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:18.485668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:18.485668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:18.485668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:18.485668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:18.485668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:18.485668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:18.485668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:18.485668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:18.491944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bb122-51b8-4be8-9b9b-f49832cd35d8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:18.495398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 377/761] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:18.502041 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef237d-f914-4bd2-95ef-4394e743e805 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:18.505193 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:18.556976 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9995bd9-8e02-405b-a2b1-78126a041e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:18.557618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9995bd9-8e02-405b-a2b1-78126a041e57 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:18.557893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9995bd9-8e02-405b-a2b1-78126a041e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:18.558160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9995bd9-8e02-405b-a2b1-78126a041e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:18.585386 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9995bd9-8e02-405b-a2b1-78126a041e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:18.587059 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9995bd9-8e02-405b-a2b1-78126a041e57 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a9995bd9-8e02-405b-a2b1-78126a041e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:18.620437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:18.621424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:18.622349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:18.622349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:18.622349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:18.622349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 385/762] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:18.628856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9995bd9-8e02-405b-a2b1-78126a041e57 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:18.690658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef237d-f914-4bd2-95ef-4394e743e805 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:18.729232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b13654-7055-4fd6-bfe7-cdb02006fdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:18.730185 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b13654-7055-4fd6-bfe7-cdb02006fdbf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:18.730669 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b13654-7055-4fd6-bfe7-cdb02006fdbf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:18.731188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b13654-7055-4fd6-bfe7-cdb02006fdbf None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:18.732177 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b13654-7055-4fd6-bfe7-cdb02006fdbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:18.732710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b13654-7055-4fd6-bfe7-cdb02006fdbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:18.734990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b13654-7055-4fd6-bfe7-cdb02006fdbf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:18.748616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 378/763] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:18.750075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b13654-7055-4fd6-bfe7-cdb02006fdbf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:18.769542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9995bd9-8e02-405b-a2b1-78126a041e57 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:18.771493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:18.812994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcc31a-bbfa-4eff-8e77-84e034b77897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:18.813749 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcc31a-bbfa-4eff-8e77-84e034b77897 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:18.814160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcc31a-bbfa-4eff-8e77-84e034b77897 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:18.814444 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcc31a-bbfa-4eff-8e77-84e034b77897 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:18.815125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcc31a-bbfa-4eff-8e77-84e034b77897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:18.815436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcc31a-bbfa-4eff-8e77-84e034b77897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:18.817056 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcc31a-bbfa-4eff-8e77-84e034b77897 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:18.822608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:18.840641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 386/764] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:18.840641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcc31a-bbfa-4eff-8e77-84e034b77897 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:18.875724 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b13654-7055-4fd6-bfe7-cdb02006fdbf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:18.914920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412496bc-c9fd-4222-9e9d-ae9af1378c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:18.915993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412496bc-c9fd-4222-9e9d-ae9af1378c34 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:18.916508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412496bc-c9fd-4222-9e9d-ae9af1378c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:18.916996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412496bc-c9fd-4222-9e9d-ae9af1378c34 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:18.917809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412496bc-c9fd-4222-9e9d-ae9af1378c34 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:18.919619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412496bc-c9fd-4222-9e9d-ae9af1378c34 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-412496bc-c9fd-4222-9e9d-ae9af1378c34 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:18.930311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:18.931437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 13:02:18.932327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:18.940837 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 379/765] 10.222.0.17 () {56 vars in 1129 bytes} [Fri Sep 9 13:02:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:18.943008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412496bc-c9fd-4222-9e9d-ae9af1378c34 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:18.962809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcc31a-bbfa-4eff-8e77-84e034b77897 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:19.004704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369bc20-5a89-4f5b-8f72-0f10407ca744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:19.005835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369bc20-5a89-4f5b-8f72-0f10407ca744 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:19.006125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369bc20-5a89-4f5b-8f72-0f10407ca744 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:19.006405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369bc20-5a89-4f5b-8f72-0f10407ca744 None admin] PATH_INFO: `/v3/projects/2b842ac074b44e148c15882a7964aa13/users/441575b8b496457b905e9d5f5846e092/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:19.008655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369bc20-5a89-4f5b-8f72-0f10407ca744 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b842ac074b44e148c15882a7964aa13, user_id=441575b8b496457b905e9d5f5846e092, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:19.010329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369bc20-5a89-4f5b-8f72-0f10407ca744 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:19.035239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 387/766] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:02:18 2022] PUT /identity/v3/projects/2b842ac074b44e148c15882a7964aa13/users/441575b8b496457b905e9d5f5846e092/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:19.036760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369bc20-5a89-4f5b-8f72-0f10407ca744 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:19.069183 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:19.078181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412496bc-c9fd-4222-9e9d-ae9af1378c34 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:19.112341 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:19.127407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0282fa3-48df-4ec6-b15c-9aaa726b64bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:19.131812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0282fa3-48df-4ec6-b15c-9aaa726b64bf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:19.133138 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0282fa3-48df-4ec6-b15c-9aaa726b64bf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:19.133627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0282fa3-48df-4ec6-b15c-9aaa726b64bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:19.144739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0282fa3-48df-4ec6-b15c-9aaa726b64bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:19.147084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0282fa3-48df-4ec6-b15c-9aaa726b64bf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:19.164177 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369bc20-5a89-4f5b-8f72-0f10407ca744 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b0282fa3-48df-4ec6-b15c-9aaa726b64bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:19.166488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:19.167448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:19.168363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:19.168363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:19.168363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:19.168363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:19.168363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:19.168363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:19.168363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:19.168363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:19.168809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 380/767] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:19.175450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c0806-33da-477a-a67a-d000541184a6 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:19.175450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c0806-33da-477a-a67a-d000541184a6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:19.175450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c0806-33da-477a-a67a-d000541184a6 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:19.176926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 381/768] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:19.182198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c0806-33da-477a-a67a-d000541184a6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:19.215775 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bbccf-2e75-47af-a1c0-bd4fbcc2fe73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:19.216792 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bbccf-2e75-47af-a1c0-bd4fbcc2fe73 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:19.217329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bbccf-2e75-47af-a1c0-bd4fbcc2fe73 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:19.217911 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bbccf-2e75-47af-a1c0-bd4fbcc2fe73 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:19.219313 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bbccf-2e75-47af-a1c0-bd4fbcc2fe73 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:19.220280 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bbccf-2e75-47af-a1c0-bd4fbcc2fe73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:19.222228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bbccf-2e75-47af-a1c0-bd4fbcc2fe73 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:19.227849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 388/769] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 9 13:02:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:19.229377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a959f-65cd-4256-a16e-6d73faac9af1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:19.229882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a959f-65cd-4256-a16e-6d73faac9af1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:19.230440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a959f-65cd-4256-a16e-6d73faac9af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:19.252563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-667a959f-65cd-4256-a16e-6d73faac9af1 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:19.254180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-667a959f-65cd-4256-a16e-6d73faac9af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:19.304932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c0806-33da-477a-a67a-d000541184a6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:19.337991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 389/770] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:19.338920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a959f-65cd-4256-a16e-6d73faac9af1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:19.358344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:19.364358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ad032-1c0c-43ff-8d31-2c6e9f377485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:19.364985 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ad032-1c0c-43ff-8d31-2c6e9f377485 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:19.365261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ad032-1c0c-43ff-8d31-2c6e9f377485 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:19.365530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ad032-1c0c-43ff-8d31-2c6e9f377485 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:19.366233 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ad032-1c0c-43ff-8d31-2c6e9f377485 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:19.366537 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ad032-1c0c-43ff-8d31-2c6e9f377485 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:19.368021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ad032-1c0c-43ff-8d31-2c6e9f377485 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:19.397131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 382/771] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:19.408194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ad032-1c0c-43ff-8d31-2c6e9f377485 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:19.454681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a959f-65cd-4256-a16e-6d73faac9af1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:19.503569 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea816faa-591d-48a5-82e7-66e178baa46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:19.504282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea816faa-591d-48a5-82e7-66e178baa46b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:19.504568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea816faa-591d-48a5-82e7-66e178baa46b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:19.504848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea816faa-591d-48a5-82e7-66e178baa46b None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:19.505499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea816faa-591d-48a5-82e7-66e178baa46b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:19.505862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea816faa-591d-48a5-82e7-66e178baa46b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:19.507267 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea816faa-591d-48a5-82e7-66e178baa46b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:19.513158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 390/772] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 9 13:02:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:19.519582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea816faa-591d-48a5-82e7-66e178baa46b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:19.548835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:19.625271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ad032-1c0c-43ff-8d31-2c6e9f377485 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:19.645480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:19.659199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea816faa-591d-48a5-82e7-66e178baa46b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:19.664500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b379592-4ecf-426c-86d6-bd533e14d836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:19.665336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b379592-4ecf-426c-86d6-bd533e14d836 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:19.665849 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b379592-4ecf-426c-86d6-bd533e14d836 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:19.666268 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b379592-4ecf-426c-86d6-bd533e14d836 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:19.667165 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b379592-4ecf-426c-86d6-bd533e14d836 None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=8d343a3db21c47e1a85a739a55559c97) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:19.667636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b379592-4ecf-426c-86d6-bd533e14d836 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:19.669231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b379592-4ecf-426c-86d6-bd533e14d836 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:19.675933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 383/773] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:19 2022] GET /identity/v3/role_assignments?user.id=8d343a3db21c47e1a85a739a55559c97&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:19.677496 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f0b96-82ea-49d2-b7a5-63d8019edd4d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:19.677932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f0b96-82ea-49d2-b7a5-63d8019edd4d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:19.678372 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f0b96-82ea-49d2-b7a5-63d8019edd4d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:19.679829 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 384/774] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:19.687100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed43df-5747-4e02-af5d-61a8a80245d2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:19.687100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed43df-5747-4e02-af5d-61a8a80245d2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:19.687100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed43df-5747-4e02-af5d-61a8a80245d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:19.712014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724c662-907c-4e68-86c1-8a012dd614c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:19.713311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724c662-907c-4e68-86c1-8a012dd614c5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:19.713838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724c662-907c-4e68-86c1-8a012dd614c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:19.714316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724c662-907c-4e68-86c1-8a012dd614c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:19.715293 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724c662-907c-4e68-86c1-8a012dd614c5 None admin] RBAC: Adding query filter params (interface=public, service_id=a8aa8125b14d48db9481d41c18c26d2a, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:19.715918 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724c662-907c-4e68-86c1-8a012dd614c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:19.716955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d9ed43df-5747-4e02-af5d-61a8a80245d2 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:19.718159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724c662-907c-4e68-86c1-8a012dd614c5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:19.718272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d9ed43df-5747-4e02-af5d-61a8a80245d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:19.728288 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 391/775] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 13:02:19 2022] GET /identity/v3/endpoints?service_id=a8aa8125b14d48db9481d41c18c26d2a&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:19.729559 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724c662-907c-4e68-86c1-8a012dd614c5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:19.806668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 385/776] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:19.808099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed43df-5747-4e02-af5d-61a8a80245d2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:19.845417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724c662-907c-4e68-86c1-8a012dd614c5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:19.881715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:19.890015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3379aa5-2750-4b52-96be-f08e28c03631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:19.891370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3379aa5-2750-4b52-96be-f08e28c03631 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:19.891656 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3379aa5-2750-4b52-96be-f08e28c03631 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:19.891977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3379aa5-2750-4b52-96be-f08e28c03631 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:19.922563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3379aa5-2750-4b52-96be-f08e28c03631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-c3379aa5-2750-4b52-96be-f08e28c03631 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:19.931131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:19.932211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:19.933080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:19.933080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:19.933080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:19.933080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:19.933080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:19.933080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:19.933080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:19.933080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:19.933080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:19.933080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:19.933080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:19.933080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:19.937830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 392/777] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:02:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:19.943703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3379aa5-2750-4b52-96be-f08e28c03631 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:19.949390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:19.975016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed43df-5747-4e02-af5d-61a8a80245d2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:20.051828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1a565-66cf-471b-9f82-7ff390f9bf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:20.052455 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1a565-66cf-471b-9f82-7ff390f9bf57 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:20.052729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1a565-66cf-471b-9f82-7ff390f9bf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:20.052997 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1a565-66cf-471b-9f82-7ff390f9bf57 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:20.053612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1a565-66cf-471b-9f82-7ff390f9bf57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:20.054978 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1a565-66cf-471b-9f82-7ff390f9bf57 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:20.063966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 386/778] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 13:02:19 2022] GET /identity/v3/services => generated 772 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:20.084891 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3379aa5-2750-4b52-96be-f08e28c03631 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:20.129755 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab8c75-f134-4ecd-a891-776be4e427c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:20.130431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab8c75-f134-4ecd-a891-776be4e427c6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:20.130717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab8c75-f134-4ecd-a891-776be4e427c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:20.130999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab8c75-f134-4ecd-a891-776be4e427c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:20.131704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab8c75-f134-4ecd-a891-776be4e427c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:20.132063 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab8c75-f134-4ecd-a891-776be4e427c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:20.133395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab8c75-f134-4ecd-a891-776be4e427c6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:20.147374 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 393/779] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:02:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:20.148505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1a565-66cf-471b-9f82-7ff390f9bf57 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:20.154583 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab8c75-f134-4ecd-a891-776be4e427c6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:20.287813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:20.340366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1a565-66cf-471b-9f82-7ff390f9bf57 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:20.341470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab8c75-f134-4ecd-a891-776be4e427c6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:20.359654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:20.382402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be256a00-5ef9-4be8-8681-ca5325bc1564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:20.382789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca4edc-08a1-4a6a-a7d8-474a6027872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:20.382973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be256a00-5ef9-4be8-8681-ca5325bc1564 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:20.383281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be256a00-5ef9-4be8-8681-ca5325bc1564 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:20.383448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca4edc-08a1-4a6a-a7d8-474a6027872d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:20.383550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be256a00-5ef9-4be8-8681-ca5325bc1564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:20.383711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca4edc-08a1-4a6a-a7d8-474a6027872d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:20.383981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca4edc-08a1-4a6a-a7d8-474a6027872d None admin] PATH_INFO: `/v3/services/network` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:20.384568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca4edc-08a1-4a6a-a7d8-474a6027872d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:20.385944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca4edc-08a1-4a6a-a7d8-474a6027872d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:20.394881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be256a00-5ef9-4be8-8681-ca5325bc1564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ffca4edc-08a1-4a6a-a7d8-474a6027872d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:20.395743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:20.396686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 13:02:20.397524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:20.398268 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be256a00-5ef9-4be8-8681-ca5325bc1564 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:20.398346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 394/780] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:20 2022] GET /identity/v3/services/network => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:20.404348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca4edc-08a1-4a6a-a7d8-474a6027872d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-be256a00-5ef9-4be8-8681-ca5325bc1564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:20.415446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:20.416411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:20.417277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:20.417277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:20.417277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:20.417277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 387/781] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:20.424264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be256a00-5ef9-4be8-8681-ca5325bc1564 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:20.555477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca4edc-08a1-4a6a-a7d8-474a6027872d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:20.606954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be256a00-5ef9-4be8-8681-ca5325bc1564 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:20.611465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:20.622835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730583e-c198-44f4-b44f-bbe676cf5880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:20.623493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730583e-c198-44f4-b44f-bbe676cf5880 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:20.623778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730583e-c198-44f4-b44f-bbe676cf5880 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:20.624058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730583e-c198-44f4-b44f-bbe676cf5880 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:20.626649 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730583e-c198-44f4-b44f-bbe676cf5880 None admin] RBAC: Adding query filter params (name=network) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:20.626976 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730583e-c198-44f4-b44f-bbe676cf5880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:20.628403 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730583e-c198-44f4-b44f-bbe676cf5880 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:20.637406 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 395/782] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 13:02:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:20.642346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730583e-c198-44f4-b44f-bbe676cf5880 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:20.672755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40add3b6-0145-498a-89e2-6bb7a0b4f979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:20.673370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40add3b6-0145-498a-89e2-6bb7a0b4f979 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:20.673652 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40add3b6-0145-498a-89e2-6bb7a0b4f979 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:20.673926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40add3b6-0145-498a-89e2-6bb7a0b4f979 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:20.679107 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40add3b6-0145-498a-89e2-6bb7a0b4f979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:20.679496 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40add3b6-0145-498a-89e2-6bb7a0b4f979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:20.680912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40add3b6-0145-498a-89e2-6bb7a0b4f979 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:20.702934 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 388/783] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:20.706364 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40add3b6-0145-498a-89e2-6bb7a0b4f979 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:20.741990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:20.820698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730583e-c198-44f4-b44f-bbe676cf5880 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:20.884592 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eb537-18fe-4353-8ea7-46f31d99d6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:20.885258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eb537-18fe-4353-8ea7-46f31d99d6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:20.885540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eb537-18fe-4353-8ea7-46f31d99d6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:20.885817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eb537-18fe-4353-8ea7-46f31d99d6e4 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:20.886503 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972eb537-18fe-4353-8ea7-46f31d99d6e4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:20.886872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972eb537-18fe-4353-8ea7-46f31d99d6e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:20.888328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972eb537-18fe-4353-8ea7-46f31d99d6e4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:20.888865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40add3b6-0145-498a-89e2-6bb7a0b4f979 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:20.893873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 396/784] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 13:02:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:20.899363 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eb537-18fe-4353-8ea7-46f31d99d6e4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:20.968195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:20.969961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dede12-3644-407e-a8f4-6e5c482b997a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:20.970784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dede12-3644-407e-a8f4-6e5c482b997a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:20.971250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dede12-3644-407e-a8f4-6e5c482b997a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:20.971678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dede12-3644-407e-a8f4-6e5c482b997a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:20.988170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dede12-3644-407e-a8f4-6e5c482b997a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:20.989838 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dede12-3644-407e-a8f4-6e5c482b997a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:21.017656 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eb537-18fe-4353-8ea7-46f31d99d6e4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-27dede12-3644-407e-a8f4-6e5c482b997a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:21.018175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:21.019321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:21.020514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:21.020514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:21.020514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:21.020514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:21.020514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:21.020514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:21.020514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:21.020514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:21.020514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 13:02:21.020514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:21.033529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 389/785] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:02:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:21.039224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dede12-3644-407e-a8f4-6e5c482b997a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:21.063105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736f246-7765-4a24-b099-82a68ac0d0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:21.063105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736f246-7765-4a24-b099-82a68ac0d0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:21.063105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736f246-7765-4a24-b099-82a68ac0d0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:21.063105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736f246-7765-4a24-b099-82a68ac0d0d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:21.063523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736f246-7765-4a24-b099-82a68ac0d0d8 None admin] RBAC: Adding query filter params (interface=public, service_id=42d6774335c04ae2877c0da52677f5be, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:21.063523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736f246-7765-4a24-b099-82a68ac0d0d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:21.064349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736f246-7765-4a24-b099-82a68ac0d0d8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:21.070725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 397/786] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 13:02:20 2022] GET /identity/v3/endpoints?service_id=42d6774335c04ae2877c0da52677f5be&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:21.076153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736f246-7765-4a24-b099-82a68ac0d0d8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:21.178948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:21.209301 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736f246-7765-4a24-b099-82a68ac0d0d8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:21.226968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dede12-3644-407e-a8f4-6e5c482b997a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:21.255381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:21.277277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92ba0f-9b2a-4f81-b19f-1847e291ea3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:21.277899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92ba0f-9b2a-4f81-b19f-1847e291ea3f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:21.278178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92ba0f-9b2a-4f81-b19f-1847e291ea3f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:21.278451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92ba0f-9b2a-4f81-b19f-1847e291ea3f None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:21.279079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92ba0f-9b2a-4f81-b19f-1847e291ea3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:21.283978 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92ba0f-9b2a-4f81-b19f-1847e291ea3f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:21.293279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 398/787] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 13:02:21 2022] GET /identity/v3/services => generated 772 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:21.297484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935094c8-737c-4c80-9ba4-d2544138e9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:21.298203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935094c8-737c-4c80-9ba4-d2544138e9ea None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:21.298548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935094c8-737c-4c80-9ba4-d2544138e9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:21.298828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935094c8-737c-4c80-9ba4-d2544138e9ea None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:21.299600 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935094c8-737c-4c80-9ba4-d2544138e9ea None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:21.299913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935094c8-737c-4c80-9ba4-d2544138e9ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:21.301463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935094c8-737c-4c80-9ba4-d2544138e9ea None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:21.323144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 390/788] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 9 13:02:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:21.332261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92ba0f-9b2a-4f81-b19f-1847e291ea3f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:21.472138 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92ba0f-9b2a-4f81-b19f-1847e291ea3f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:21.513337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b57b12-329f-48ad-814b-45f98fc02b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:21.514010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b57b12-329f-48ad-814b-45f98fc02b3b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:21.514287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b57b12-329f-48ad-814b-45f98fc02b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:21.514557 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b57b12-329f-48ad-814b-45f98fc02b3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:21.528164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b57b12-329f-48ad-814b-45f98fc02b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:21.529669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b57b12-329f-48ad-814b-45f98fc02b3b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-75b57b12-329f-48ad-814b-45f98fc02b3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:21.546767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:21.549297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:21.551358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:21.551358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:21.551358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:21.551358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 399/789] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:21.557600 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935094c8-737c-4c80-9ba4-d2544138e9ea None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:21.647403 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237efe1-5c34-4faa-a1d0-133d6f332e73 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:21.647717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237efe1-5c34-4faa-a1d0-133d6f332e73 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:21.647961 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237efe1-5c34-4faa-a1d0-133d6f332e73 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:21.649585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 400/790] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:02:21.654886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c718c8-e0d2-4cb6-b903-7d8c2687acb7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:21.655213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c718c8-e0d2-4cb6-b903-7d8c2687acb7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:21.655628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c718c8-e0d2-4cb6-b903-7d8c2687acb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:21.674863 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-85c718c8-e0d2-4cb6-b903-7d8c2687acb7 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:21.676105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-85c718c8-e0d2-4cb6-b903-7d8c2687acb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:21.708751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:21.753097 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935094c8-737c-4c80-9ba4-d2544138e9ea None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:21.791191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 401/791] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:21.816037 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d145184-6dff-4ce7-9895-068d05b9c068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:21.816722 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d145184-6dff-4ce7-9895-068d05b9c068 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:21.817006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d145184-6dff-4ce7-9895-068d05b9c068 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:21.817282 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d145184-6dff-4ce7-9895-068d05b9c068 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:21.817933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d145184-6dff-4ce7-9895-068d05b9c068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:21.818292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d145184-6dff-4ce7-9895-068d05b9c068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:21.819774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d145184-6dff-4ce7-9895-068d05b9c068 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:21.823344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:21.840926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 391/792] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:21.847206 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c718c8-e0d2-4cb6-b903-7d8c2687acb7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:21.998137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c718c8-e0d2-4cb6-b903-7d8c2687acb7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:22.049778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d0927-51b5-43eb-b8c5-665e04812a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:22.050750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d0927-51b5-43eb-b8c5-665e04812a99 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:22.051274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d0927-51b5-43eb-b8c5-665e04812a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:22.051739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d0927-51b5-43eb-b8c5-665e04812a99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:22.061789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d0927-51b5-43eb-b8c5-665e04812a99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:22.063882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d0927-51b5-43eb-b8c5-665e04812a99 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-270d0927-51b5-43eb-b8c5-665e04812a99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:22.078751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:22.079807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:22.080706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:22.080706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:22.080706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:22.080706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:22.080706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:22.080706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:22.080706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:22.080706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:22.083013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 402/793] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:22.088930 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d145184-6dff-4ce7-9895-068d05b9c068 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:22.231816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d145184-6dff-4ce7-9895-068d05b9c068 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:22.250796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc26e69-91f3-4aaf-ad4c-a7d1f82e892d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:22.251226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc26e69-91f3-4aaf-ad4c-a7d1f82e892d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:22.251482 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc26e69-91f3-4aaf-ad4c-a7d1f82e892d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:22.253100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 403/794] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:22.259185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5399c70-bfcd-4d44-8f1d-57f6365b248d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:22.259477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5399c70-bfcd-4d44-8f1d-57f6365b248d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:22.259750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5399c70-bfcd-4d44-8f1d-57f6365b248d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:22.279964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b5399c70-bfcd-4d44-8f1d-57f6365b248d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:22.281777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b5399c70-bfcd-4d44-8f1d-57f6365b248d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:22.282067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc14c3-ecb8-4a93-8b33-3eb038682f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:22.282698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc14c3-ecb8-4a93-8b33-3eb038682f0e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:22.282976 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc14c3-ecb8-4a93-8b33-3eb038682f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:22.283297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc14c3-ecb8-4a93-8b33-3eb038682f0e None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:22.284271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc14c3-ecb8-4a93-8b33-3eb038682f0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:22.285495 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc14c3-ecb8-4a93-8b33-3eb038682f0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:22.287040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc14c3-ecb8-4a93-8b33-3eb038682f0e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:22.304625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 392/795] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:22.305400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc14c3-ecb8-4a93-8b33-3eb038682f0e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:22.313700 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:22.367645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 404/796] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:22.371916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5399c70-bfcd-4d44-8f1d-57f6365b248d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:22.400722 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:22.478110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5399c70-bfcd-4d44-8f1d-57f6365b248d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:22.513833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc725cf3-accc-4a9b-91a3-d06ac686f1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:22.514840 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725cf3-accc-4a9b-91a3-d06ac686f1a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:22.516080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725cf3-accc-4a9b-91a3-d06ac686f1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:22.516560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725cf3-accc-4a9b-91a3-d06ac686f1a0 None admin] PATH_INFO: `/v3/projects/2b842ac074b44e148c15882a7964aa13/users/c361a3e83f8e45c2b4d28f97f0a9d069/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:22.518715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc725cf3-accc-4a9b-91a3-d06ac686f1a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b842ac074b44e148c15882a7964aa13, user_id=c361a3e83f8e45c2b4d28f97f0a9d069, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:22.520463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc725cf3-accc-4a9b-91a3-d06ac686f1a0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:22.535868 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 405/797] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:02:22 2022] PUT /identity/v3/projects/2b842ac074b44e148c15882a7964aa13/users/c361a3e83f8e45c2b4d28f97f0a9d069/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:22.537027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc14c3-ecb8-4a93-8b33-3eb038682f0e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:22.603700 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c74b2-157e-4431-9b36-3cf673c9a46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:22.604267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c74b2-157e-4431-9b36-3cf673c9a46a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:22.604540 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c74b2-157e-4431-9b36-3cf673c9a46a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:22.604805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c74b2-157e-4431-9b36-3cf673c9a46a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:22.623279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c74b2-157e-4431-9b36-3cf673c9a46a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-785c74b2-157e-4431-9b36-3cf673c9a46a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:22.628719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:22.629871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:22.630866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:22.630866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:22.630866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:22.630866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:22.630866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:22.630866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:22.630866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:22.630866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:22.630866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:22.630866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:22.630866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:22.630866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:22.634011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 393/798] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:02:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:22.640972 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc725cf3-accc-4a9b-91a3-d06ac686f1a0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:22.667529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:22.697109 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c74b2-157e-4431-9b36-3cf673c9a46a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:22.821929 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc725cf3-accc-4a9b-91a3-d06ac686f1a0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:22.854640 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c74b2-157e-4431-9b36-3cf673c9a46a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:22.861672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5c7c2-aa51-4fa0-9d79-a1c989f4a5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:22.862303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5c7c2-aa51-4fa0-9d79-a1c989f4a5ae None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:22.862581 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5c7c2-aa51-4fa0-9d79-a1c989f4a5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:22.862854 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5c7c2-aa51-4fa0-9d79-a1c989f4a5ae None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:22.864182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5c7c2-aa51-4fa0-9d79-a1c989f4a5ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:22.864524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5c7c2-aa51-4fa0-9d79-a1c989f4a5ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:22.866291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5c7c2-aa51-4fa0-9d79-a1c989f4a5ae None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:22.870078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:22.880453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 406/799] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:02:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:22.881813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4c194-109b-459c-80b4-ed964ff5d4e9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:22.881813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4c194-109b-459c-80b4-ed964ff5d4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:22.882431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4c194-109b-459c-80b4-ed964ff5d4e9 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:22.883684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 407/800] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:22.887706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4c194-109b-459c-80b4-ed964ff5d4e9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:22.902868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce99547-0543-42e2-a7fc-495465b0f13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:22.903330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce99547-0543-42e2-a7fc-495465b0f13f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:22.903613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce99547-0543-42e2-a7fc-495465b0f13f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:22.903890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce99547-0543-42e2-a7fc-495465b0f13f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:22.915436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce99547-0543-42e2-a7fc-495465b0f13f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4ce99547-0543-42e2-a7fc-495465b0f13f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:22.920680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:22.922208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 394/801] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:02:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:22.923380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7de01-fdb6-4336-b438-8852ca527b43 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:22.924245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7de01-fdb6-4336-b438-8852ca527b43 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:22.924245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7de01-fdb6-4336-b438-8852ca527b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:22.945084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b1c7de01-fdb6-4336-b438-8852ca527b43 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:22.946692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b1c7de01-fdb6-4336-b438-8852ca527b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:23.035369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 395/802] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:23.036599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7de01-fdb6-4336-b438-8852ca527b43 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:23.123152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:23.164286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4c194-109b-459c-80b4-ed964ff5d4e9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:23.216416 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7de01-fdb6-4336-b438-8852ca527b43 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:23.219720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:23.233262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b348faf-1de0-4193-bca5-b9eff2791bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:23.234424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b348faf-1de0-4193-bca5-b9eff2791bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:23.234707 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b348faf-1de0-4193-bca5-b9eff2791bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:23.234961 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b348faf-1de0-4193-bca5-b9eff2791bf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:23.255816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b348faf-1de0-4193-bca5-b9eff2791bf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:23.257259 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b348faf-1de0-4193-bca5-b9eff2791bf6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-2b348faf-1de0-4193-bca5-b9eff2791bf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:23.275925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:23.276937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:23.277828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:23.277828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:23.277828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:23.277828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 408/803] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:23.284549 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b348faf-1de0-4193-bca5-b9eff2791bf6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:23.301002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f659be6-3934-4e5a-acef-beabd030c4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:23.301631 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f659be6-3934-4e5a-acef-beabd030c4dc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:23.301913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f659be6-3934-4e5a-acef-beabd030c4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:23.302221 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f659be6-3934-4e5a-acef-beabd030c4dc None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:23.302898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f659be6-3934-4e5a-acef-beabd030c4dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:23.303230 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f659be6-3934-4e5a-acef-beabd030c4dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:23.304603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f659be6-3934-4e5a-acef-beabd030c4dc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:23.329338 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 396/804] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:02:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:23.334438 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f659be6-3934-4e5a-acef-beabd030c4dc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:23.453402 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b348faf-1de0-4193-bca5-b9eff2791bf6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:23.484868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f659be6-3934-4e5a-acef-beabd030c4dc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:23.489009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:23.494993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16652ae7-4ee5-48fc-8ca3-fd4e1774747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:23.496239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16652ae7-4ee5-48fc-8ca3-fd4e1774747b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:23.496690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16652ae7-4ee5-48fc-8ca3-fd4e1774747b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:23.497038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16652ae7-4ee5-48fc-8ca3-fd4e1774747b None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:23.497758 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16652ae7-4ee5-48fc-8ca3-fd4e1774747b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:23.498156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16652ae7-4ee5-48fc-8ca3-fd4e1774747b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:23.500474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16652ae7-4ee5-48fc-8ca3-fd4e1774747b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:23.518664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 409/805] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:23.518664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16652ae7-4ee5-48fc-8ca3-fd4e1774747b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:23.531387 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b61dcb-ebc9-4419-b1ed-4ac0883bed02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:23.532414 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b61dcb-ebc9-4419-b1ed-4ac0883bed02 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:23.533035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b61dcb-ebc9-4419-b1ed-4ac0883bed02 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:23.533671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b61dcb-ebc9-4419-b1ed-4ac0883bed02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:23.545313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b61dcb-ebc9-4419-b1ed-4ac0883bed02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:23.547850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b61dcb-ebc9-4419-b1ed-4ac0883bed02 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-56b61dcb-ebc9-4419-b1ed-4ac0883bed02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:23.570617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:23.571677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:23.572580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:23.572580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:23.572580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:23.574843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 397/806] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:23.576053 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b61dcb-ebc9-4419-b1ed-4ac0883bed02 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:23.599020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:23.637697 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16652ae7-4ee5-48fc-8ca3-fd4e1774747b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:23.677715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590bb86-09c8-4b02-b7de-92b22899508d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:23.678361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590bb86-09c8-4b02-b7de-92b22899508d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:23.678644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590bb86-09c8-4b02-b7de-92b22899508d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:23.678931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590bb86-09c8-4b02-b7de-92b22899508d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:23.679526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590bb86-09c8-4b02-b7de-92b22899508d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:23.680890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590bb86-09c8-4b02-b7de-92b22899508d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-5590bb86-09c8-4b02-b7de-92b22899508d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:23.691570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:23.692532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 13:02:23.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:23.697617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 410/807] 10.222.0.17 () {56 vars in 1129 bytes} [Fri Sep 9 13:02:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:23.698897 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590bb86-09c8-4b02-b7de-92b22899508d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:23.701352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b61dcb-ebc9-4419-b1ed-4ac0883bed02 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:23.740305 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cd16a-82fd-486d-830a-5f6961fa2ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:23.741288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cd16a-82fd-486d-830a-5f6961fa2ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:23.741763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cd16a-82fd-486d-830a-5f6961fa2ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:23.742245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cd16a-82fd-486d-830a-5f6961fa2ab0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:23.752170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cd16a-82fd-486d-830a-5f6961fa2ab0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:23.753982 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cd16a-82fd-486d-830a-5f6961fa2ab0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:23.761992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-103cd16a-82fd-486d-830a-5f6961fa2ab0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:23.769192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:23.770145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:23.771103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:23.771103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:23.771103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:23.771103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:23.771103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:23.771103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:23.771103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:23.771103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:23.771103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:02:23.771103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:23.778249 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 398/808] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 13:02:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:23.779342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cd16a-82fd-486d-830a-5f6961fa2ab0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:23.815336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590bb86-09c8-4b02-b7de-92b22899508d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:23.855354 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9d16d-15a8-4205-8c97-c88dd3d53e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:23.855971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9d16d-15a8-4205-8c97-c88dd3d53e3e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:23.856253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9d16d-15a8-4205-8c97-c88dd3d53e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:23.856529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9d16d-15a8-4205-8c97-c88dd3d53e3e None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:23.857170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9d16d-15a8-4205-8c97-c88dd3d53e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:23.857478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9d16d-15a8-4205-8c97-c88dd3d53e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:23.859421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9d16d-15a8-4205-8c97-c88dd3d53e3e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:23.872675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 411/809] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:23.874041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9d16d-15a8-4205-8c97-c88dd3d53e3e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:23.878721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:23.905237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cd16a-82fd-486d-830a-5f6961fa2ab0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:23.944261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee6037-bdf6-4bf9-88ef-ec7f362f6d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:23.945020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee6037-bdf6-4bf9-88ef-ec7f362f6d6e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:23.945296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee6037-bdf6-4bf9-88ef-ec7f362f6d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:23.945574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee6037-bdf6-4bf9-88ef-ec7f362f6d6e None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:23.946183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee6037-bdf6-4bf9-88ef-ec7f362f6d6e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:23.946487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee6037-bdf6-4bf9-88ef-ec7f362f6d6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:23.947882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee6037-bdf6-4bf9-88ef-ec7f362f6d6e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:23.957077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 399/810] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 9 13:02:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:23.957077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee6037-bdf6-4bf9-88ef-ec7f362f6d6e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:23.992736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9d16d-15a8-4205-8c97-c88dd3d53e3e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:24.010220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:24.030229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171792fa-7334-4506-8bfc-dacf1b4b935c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:24.031182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171792fa-7334-4506-8bfc-dacf1b4b935c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:24.031634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171792fa-7334-4506-8bfc-dacf1b4b935c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:24.031931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171792fa-7334-4506-8bfc-dacf1b4b935c None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:24.032632 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171792fa-7334-4506-8bfc-dacf1b4b935c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:24.032931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171792fa-7334-4506-8bfc-dacf1b4b935c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:24.034339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171792fa-7334-4506-8bfc-dacf1b4b935c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:24.053240 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 412/811] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 13:02:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:24.058743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95b96c-8b89-498a-8f69-2d15dd6ce1f9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:24.059026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95b96c-8b89-498a-8f69-2d15dd6ce1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:24.059336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95b96c-8b89-498a-8f69-2d15dd6ce1f9 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:24.060573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 413/812] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:24.061584 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95b96c-8b89-498a-8f69-2d15dd6ce1f9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:24.068085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee6037-bdf6-4bf9-88ef-ec7f362f6d6e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:24.103808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccddd3-aa71-443d-84ea-2c0c74f031f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:24.104440 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccddd3-aa71-443d-84ea-2c0c74f031f6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:24.104717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccddd3-aa71-443d-84ea-2c0c74f031f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:24.104985 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccddd3-aa71-443d-84ea-2c0c74f031f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:24.113287 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccddd3-aa71-443d-84ea-2c0c74f031f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:24.114747 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccddd3-aa71-443d-84ea-2c0c74f031f6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3cccddd3-aa71-443d-84ea-2c0c74f031f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:24.125968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:24.127117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:24.128306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:24.128306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:24.128306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:24.128306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:24.128306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:24.128306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:24.128306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:24.128306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:24.128306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 13:02:24.128306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:24.132156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 400/813] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 13:02:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:24.132983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccddd3-aa71-443d-84ea-2c0c74f031f6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:24.135791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:24.177183 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95b96c-8b89-498a-8f69-2d15dd6ce1f9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:24.213973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea79b6-1260-4166-aca2-b0fb5c1cb8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:24.214572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea79b6-1260-4166-aca2-b0fb5c1cb8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:24.214845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea79b6-1260-4166-aca2-b0fb5c1cb8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:24.215139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea79b6-1260-4166-aca2-b0fb5c1cb8b0 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:24.215752 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea79b6-1260-4166-aca2-b0fb5c1cb8b0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:24.216050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea79b6-1260-4166-aca2-b0fb5c1cb8b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:24.217618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea79b6-1260-4166-aca2-b0fb5c1cb8b0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:24.223431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 414/814] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 9 13:02:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:24.224777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc2459-72b7-42c4-b896-378f6f4242b8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:24.225055 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc2459-72b7-42c4-b896-378f6f4242b8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:24.225319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc2459-72b7-42c4-b896-378f6f4242b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:24.240060 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccddd3-aa71-443d-84ea-2c0c74f031f6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:24.244716 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-23bc2459-72b7-42c4-b896-378f6f4242b8 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:24.245836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-23bc2459-72b7-42c4-b896-378f6f4242b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:24.273249 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dad15f-1b2a-4e50-9186-c2e397d8ddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:24.273874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dad15f-1b2a-4e50-9186-c2e397d8ddd7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:24.274138 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dad15f-1b2a-4e50-9186-c2e397d8ddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:24.274408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dad15f-1b2a-4e50-9186-c2e397d8ddd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:24.278421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:24.320877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 415/815] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:24.322045 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc2459-72b7-42c4-b896-378f6f4242b8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:24.353478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dad15f-1b2a-4e50-9186-c2e397d8ddd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:24.355353 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dad15f-1b2a-4e50-9186-c2e397d8ddd7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:24.357605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:24.463270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc2459-72b7-42c4-b896-378f6f4242b8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:24.473529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 401/816] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:02:24.474623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dad15f-1b2a-4e50-9186-c2e397d8ddd7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:24.508641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ba570-e035-4d72-a82b-2438b5104e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:24.509487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ba570-e035-4d72-a82b-2438b5104e5f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:24.509962 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ba570-e035-4d72-a82b-2438b5104e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:24.510476 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ba570-e035-4d72-a82b-2438b5104e5f None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:24.511411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ba570-e035-4d72-a82b-2438b5104e5f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:24.511862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ba570-e035-4d72-a82b-2438b5104e5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:24.513633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ba570-e035-4d72-a82b-2438b5104e5f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:24.537754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 416/817] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 9 13:02:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:24.538911 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ba570-e035-4d72-a82b-2438b5104e5f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:24.571281 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:24.601099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dad15f-1b2a-4e50-9186-c2e397d8ddd7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:24.603091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:24.650150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30807d0f-46be-427f-ba5a-83d83bb672b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:24.651036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30807d0f-46be-427f-ba5a-83d83bb672b5 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:24.651345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30807d0f-46be-427f-ba5a-83d83bb672b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:24.651620 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30807d0f-46be-427f-ba5a-83d83bb672b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:24.652931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30807d0f-46be-427f-ba5a-83d83bb672b5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:24.654568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30807d0f-46be-427f-ba5a-83d83bb672b5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:24.655649 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ba570-e035-4d72-a82b-2438b5104e5f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:24.679301 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 402/818] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:24.680136 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30807d0f-46be-427f-ba5a-83d83bb672b5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:24.708219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cde41-61ce-48ed-b358-739828dcb983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:24.709371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cde41-61ce-48ed-b358-739828dcb983 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:24.709866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cde41-61ce-48ed-b358-739828dcb983 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:24.710372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cde41-61ce-48ed-b358-739828dcb983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:24.723584 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cde41-61ce-48ed-b358-739828dcb983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:24.726567 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cde41-61ce-48ed-b358-739828dcb983 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d88cde41-61ce-48ed-b358-739828dcb983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:24.744902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:24.745998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:24.746945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:24.746945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:24.746945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:24.749992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 417/819] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:24.751292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cde41-61ce-48ed-b358-739828dcb983 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:24.813031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30807d0f-46be-427f-ba5a-83d83bb672b5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:24.856740 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f521d-af9b-4c0a-908b-86e701d53057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:24.857413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f521d-af9b-4c0a-908b-86e701d53057 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:24.857699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f521d-af9b-4c0a-908b-86e701d53057 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:24.857980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f521d-af9b-4c0a-908b-86e701d53057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:24.863385 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:24.873211 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f521d-af9b-4c0a-908b-86e701d53057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:24.875089 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f521d-af9b-4c0a-908b-86e701d53057 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5b2f521d-af9b-4c0a-908b-86e701d53057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:24.895078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:24.896744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:24.897971 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:24.897971 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:24.897971 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:24.897971 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 403/820] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:24.897971 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f521d-af9b-4c0a-908b-86e701d53057 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:24.905577 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cde41-61ce-48ed-b358-739828dcb983 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:24.916219 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:24.961585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b6aad-72e8-4e90-ab0f-9c4a274129e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:24.962672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b6aad-72e8-4e90-ab0f-9c4a274129e3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:24.963252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b6aad-72e8-4e90-ab0f-9c4a274129e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:24.964296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b6aad-72e8-4e90-ab0f-9c4a274129e3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:24.965115 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b6aad-72e8-4e90-ab0f-9c4a274129e3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:24.967136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b6aad-72e8-4e90-ab0f-9c4a274129e3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-1d2b6aad-72e8-4e90-ab0f-9c4a274129e3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:24.979816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:24.980872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 13:02:24.981940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:24.984126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 418/821] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:24 2022] GET /identity/v3/services/network => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:24.985210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b6aad-72e8-4e90-ab0f-9c4a274129e3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:25.017504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f521d-af9b-4c0a-908b-86e701d53057 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:25.055236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1e135-6559-4b16-bb4a-e999471edc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:25.055969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1e135-6559-4b16-bb4a-e999471edc63 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.056250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1e135-6559-4b16-bb4a-e999471edc63 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.056557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1e135-6559-4b16-bb4a-e999471edc63 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.057279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1e135-6559-4b16-bb4a-e999471edc63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:25.057587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1e135-6559-4b16-bb4a-e999471edc63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:25.058977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1e135-6559-4b16-bb4a-e999471edc63 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:25.077110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 404/822] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:25.077110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460865ef-ed9c-4f8e-84d6-d0c9788c4166 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.077110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460865ef-ed9c-4f8e-84d6-d0c9788c4166 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.077110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460865ef-ed9c-4f8e-84d6-d0c9788c4166 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.077110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 405/823] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:25.077110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460865ef-ed9c-4f8e-84d6-d0c9788c4166 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:25.099619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b6aad-72e8-4e90-ab0f-9c4a274129e3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:25.134263 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ff3d8-f80e-41f5-b6d3-d752fda6806c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:25.134874 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ff3d8-f80e-41f5-b6d3-d752fda6806c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.135214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ff3d8-f80e-41f5-b6d3-d752fda6806c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.135454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ff3d8-f80e-41f5-b6d3-d752fda6806c None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.136092 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ff3d8-f80e-41f5-b6d3-d752fda6806c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:25.136395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ff3d8-f80e-41f5-b6d3-d752fda6806c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:25.137786 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ff3d8-f80e-41f5-b6d3-d752fda6806c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:25.145347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:25.151451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 419/824] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:25.151451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ff3d8-f80e-41f5-b6d3-d752fda6806c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:25.160111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:25.195097 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460865ef-ed9c-4f8e-84d6-d0c9788c4166 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:25.229831 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562059a0-b655-48e1-a07d-c3a38856f60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:25.230449 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562059a0-b655-48e1-a07d-c3a38856f60b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.230728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562059a0-b655-48e1-a07d-c3a38856f60b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.231000 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562059a0-b655-48e1-a07d-c3a38856f60b None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.231663 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562059a0-b655-48e1-a07d-c3a38856f60b None admin] RBAC: Adding query filter params (name=network) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:25.231969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562059a0-b655-48e1-a07d-c3a38856f60b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:25.233334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562059a0-b655-48e1-a07d-c3a38856f60b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:25.238272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 406/825] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 13:02:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:25.239553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77aab8-a2a3-4068-845a-60bdeeb09c5f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.239841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77aab8-a2a3-4068-845a-60bdeeb09c5f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.240114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77aab8-a2a3-4068-845a-60bdeeb09c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.258367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1a77aab8-a2a3-4068-845a-60bdeeb09c5f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:25.259605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1a77aab8-a2a3-4068-845a-60bdeeb09c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:25.267245 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ff3d8-f80e-41f5-b6d3-d752fda6806c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:25.303020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90935f-0199-4563-a2a6-90aeb0bac0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:25.303996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90935f-0199-4563-a2a6-90aeb0bac0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.304485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90935f-0199-4563-a2a6-90aeb0bac0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.304937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90935f-0199-4563-a2a6-90aeb0bac0e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.312939 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90935f-0199-4563-a2a6-90aeb0bac0e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:25.314767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90935f-0199-4563-a2a6-90aeb0bac0e8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-fe90935f-0199-4563-a2a6-90aeb0bac0e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:25.326484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:25.328070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:25.328998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:25.328998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:25.328998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:25.328998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:25.328998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:25.328998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:25.328998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:25.328998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:25.329433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 407/826] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:25.329433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77aab8-a2a3-4068-845a-60bdeeb09c5f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:25.330736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 420/827] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:25.331714 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90935f-0199-4563-a2a6-90aeb0bac0e8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:25.404025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:25.404230 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:25.446937 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77aab8-a2a3-4068-845a-60bdeeb09c5f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:25.453193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90935f-0199-4563-a2a6-90aeb0bac0e8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:25.482777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60083db-c1de-453c-a56f-9b92efea155a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:25.484296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60083db-c1de-453c-a56f-9b92efea155a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.484786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60083db-c1de-453c-a56f-9b92efea155a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.485178 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60083db-c1de-453c-a56f-9b92efea155a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.491964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d93463-b435-48e3-a9bf-5ee48678a343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:25.492798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60083db-c1de-453c-a56f-9b92efea155a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:25.493267 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d93463-b435-48e3-a9bf-5ee48678a343 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.493708 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d93463-b435-48e3-a9bf-5ee48678a343 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.494126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d93463-b435-48e3-a9bf-5ee48678a343 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.494618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60083db-c1de-453c-a56f-9b92efea155a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:25.495097 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d93463-b435-48e3-a9bf-5ee48678a343 None admin] RBAC: Adding query filter params (type=network) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:25.495543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d93463-b435-48e3-a9bf-5ee48678a343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:25.497188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d93463-b435-48e3-a9bf-5ee48678a343 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:25.503686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 421/828] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 13:02:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:25.504709 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d93463-b435-48e3-a9bf-5ee48678a343 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d60083db-c1de-453c-a56f-9b92efea155a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:25.512907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:25.514006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:25.515188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:25.515188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:25.515188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:25.515188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:25.515188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:25.515188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:25.515188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:25.515188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:25.517467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 408/829] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:25.518454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60083db-c1de-453c-a56f-9b92efea155a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:25.642297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d93463-b435-48e3-a9bf-5ee48678a343 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:25.660028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60083db-c1de-453c-a56f-9b92efea155a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:25.666840 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:25.681583 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b5315-a094-47eb-baa9-12a8d3523507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:25.683005 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b5315-a094-47eb-baa9-12a8d3523507 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.683613 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b5315-a094-47eb-baa9-12a8d3523507 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.684088 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b5315-a094-47eb-baa9-12a8d3523507 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.684949 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b5315-a094-47eb-baa9-12a8d3523507 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:25.685408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b5315-a094-47eb-baa9-12a8d3523507 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:25.686954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:25.687379 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b5315-a094-47eb-baa9-12a8d3523507 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:25.698363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1f30a-8fc9-46da-b993-dcb9c2ebbf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:25.698968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1f30a-8fc9-46da-b993-dcb9c2ebbf36 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.699274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1f30a-8fc9-46da-b993-dcb9c2ebbf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.699540 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1f30a-8fc9-46da-b993-dcb9c2ebbf36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.700203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1f30a-8fc9-46da-b993-dcb9c2ebbf36 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:25.707027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 422/830] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:25.707180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1f30a-8fc9-46da-b993-dcb9c2ebbf36 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:25.707259 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b5315-a094-47eb-baa9-12a8d3523507 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:25.735082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 409/831] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:25 2022] POST /identity/v3/endpoints => generated 337 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:25.735082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1f30a-8fc9-46da-b993-dcb9c2ebbf36 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:25.847898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b5315-a094-47eb-baa9-12a8d3523507 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:25.873554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1f30a-8fc9-46da-b993-dcb9c2ebbf36 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:25.898082 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651397b-11a7-4812-a88f-fef13061cb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:25.898701 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651397b-11a7-4812-a88f-fef13061cb9d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.899286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651397b-11a7-4812-a88f-fef13061cb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.899563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651397b-11a7-4812-a88f-fef13061cb9d None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.900260 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651397b-11a7-4812-a88f-fef13061cb9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:25.900834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651397b-11a7-4812-a88f-fef13061cb9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:25.902726 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651397b-11a7-4812-a88f-fef13061cb9d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:25.916100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833346b-0144-4792-a204-60f8e6928fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:25.916757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833346b-0144-4792-a204-60f8e6928fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.917124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833346b-0144-4792-a204-60f8e6928fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.917423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833346b-0144-4792-a204-60f8e6928fc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.917526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 423/832] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:25.918335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651397b-11a7-4812-a88f-fef13061cb9d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:25.934864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833346b-0144-4792-a204-60f8e6928fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8833346b-0144-4792-a204-60f8e6928fc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:25.943518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:25.944492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:25.945333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:25.945333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:25.945333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:25.945333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:25.945333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:25.945333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:25.945333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:25.945333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:25.945333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:25.945333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:25.945333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:25.945333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:25.945333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 410/833] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:02:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:25.947346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833346b-0144-4792-a204-60f8e6928fc3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:25.982629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:25.996555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:26.055017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651397b-11a7-4812-a88f-fef13061cb9d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:26.060968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833346b-0144-4792-a204-60f8e6928fc3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:26.094479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019faeb-0779-4bcc-a04c-d482ae664e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:26.095116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019faeb-0779-4bcc-a04c-d482ae664e87 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:26.095401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019faeb-0779-4bcc-a04c-d482ae664e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:26.095675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019faeb-0779-4bcc-a04c-d482ae664e87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:26.096399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019faeb-0779-4bcc-a04c-d482ae664e87 None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=441575b8b496457b905e9d5f5846e092) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:26.096710 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019faeb-0779-4bcc-a04c-d482ae664e87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:26.097575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73bdd8-5664-4071-8d50-f62b81095a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:26.098142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019faeb-0779-4bcc-a04c-d482ae664e87 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:26.098243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73bdd8-5664-4071-8d50-f62b81095a6d None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:26.098490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73bdd8-5664-4071-8d50-f62b81095a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:26.098762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73bdd8-5664-4071-8d50-f62b81095a6d None admin] PATH_INFO: `/v3/projects/2b842ac074b44e148c15882a7964aa13/users/8d343a3db21c47e1a85a739a55559c97/roles/6e17076d694e40c29dd5e2dbc6998270` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:26.111232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 424/834] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:25 2022] GET /identity/v3/role_assignments?user.id=441575b8b496457b905e9d5f5846e092&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:26.111232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019faeb-0779-4bcc-a04c-d482ae664e87 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:26.119145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73bdd8-5664-4071-8d50-f62b81095a6d None admin] RBAC: Authorizing `identity:create_grant(project_id=2b842ac074b44e148c15882a7964aa13, user_id=8d343a3db21c47e1a85a739a55559c97, role_id=6e17076d694e40c29dd5e2dbc6998270)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:26.120768 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73bdd8-5664-4071-8d50-f62b81095a6d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:26.147177 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 411/835] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:02:25 2022] PUT /identity/v3/projects/2b842ac074b44e148c15882a7964aa13/users/8d343a3db21c47e1a85a739a55559c97/roles/6e17076d694e40c29dd5e2dbc6998270 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:26.244421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019faeb-0779-4bcc-a04c-d482ae664e87 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:26.256515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:26.294981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20050bc-4b5f-4f92-95a8-1b43f70cea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:26.295707 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20050bc-4b5f-4f92-95a8-1b43f70cea1b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:26.296184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20050bc-4b5f-4f92-95a8-1b43f70cea1b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:26.296686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20050bc-4b5f-4f92-95a8-1b43f70cea1b None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:26.297536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20050bc-4b5f-4f92-95a8-1b43f70cea1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:26.297896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20050bc-4b5f-4f92-95a8-1b43f70cea1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:26.299308 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20050bc-4b5f-4f92-95a8-1b43f70cea1b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:26.309283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 425/836] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:02:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:26.316236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73bdd8-5664-4071-8d50-f62b81095a6d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:26.410779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:26.427291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73bdd8-5664-4071-8d50-f62b81095a6d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:26.465276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f17ccd-5239-4e70-b662-058ceb4eae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:26.465914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17ccd-5239-4e70-b662-058ceb4eae15 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:26.466186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17ccd-5239-4e70-b662-058ceb4eae15 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:26.466450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17ccd-5239-4e70-b662-058ceb4eae15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:26.477973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f17ccd-5239-4e70-b662-058ceb4eae15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:26.479452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f17ccd-5239-4e70-b662-058ceb4eae15 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-40f17ccd-5239-4e70-b662-058ceb4eae15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:26.495730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:26.496715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:26.497623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:26.497623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:26.497623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:26.497623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 412/837] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:26.502767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20050bc-4b5f-4f92-95a8-1b43f70cea1b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:26.616005 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20050bc-4b5f-4f92-95a8-1b43f70cea1b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:26.652351 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88b240-b0a5-46d6-8464-b77f45df8e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:26.653019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88b240-b0a5-46d6-8464-b77f45df8e76 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:26.653293 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88b240-b0a5-46d6-8464-b77f45df8e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:26.653564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88b240-b0a5-46d6-8464-b77f45df8e76 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:26.654250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88b240-b0a5-46d6-8464-b77f45df8e76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:26.654553 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88b240-b0a5-46d6-8464-b77f45df8e76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:26.656913 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88b240-b0a5-46d6-8464-b77f45df8e76 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:26.669673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 426/838] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:26.677038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f17ccd-5239-4e70-b662-058ceb4eae15 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:26.786026 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f17ccd-5239-4e70-b662-058ceb4eae15 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:26.822480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea68414-0897-4dbc-a6af-02d0c0cde49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:26.823138 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea68414-0897-4dbc-a6af-02d0c0cde49b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:26.823415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea68414-0897-4dbc-a6af-02d0c0cde49b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:26.823682 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea68414-0897-4dbc-a6af-02d0c0cde49b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:26.833437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea68414-0897-4dbc-a6af-02d0c0cde49b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:26.834995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea68414-0897-4dbc-a6af-02d0c0cde49b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:26.837178 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-fea68414-0897-4dbc-a6af-02d0c0cde49b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:26.850502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:26.851598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:26.852550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:26.852550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:26.852550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:26.852550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:26.852550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:26.852550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:26.852550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:26.852550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:26.852550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 13:02:26.852550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:26.852550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 413/839] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:02:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:26.858101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88b240-b0a5-46d6-8464-b77f45df8e76 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:26.889528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:26.957670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee19c9-b61c-4358-a85a-4bfcd5ee1d68 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:26.958176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee19c9-b61c-4358-a85a-4bfcd5ee1d68 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:26.958176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee19c9-b61c-4358-a85a-4bfcd5ee1d68 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:26.959791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 414/840] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:26.968449 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745c333-0627-40c4-b158-b465b97254ab None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:26.968836 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745c333-0627-40c4-b158-b465b97254ab None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:26.969164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745c333-0627-40c4-b158-b465b97254ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:26.981061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88b240-b0a5-46d6-8464-b77f45df8e76 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:26.990377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2745c333-0627-40c4-b158-b465b97254ab None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:26.991913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2745c333-0627-40c4-b158-b465b97254ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:27.024401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b75392-a291-4592-b89a-35778fbe3045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:27.024989 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b75392-a291-4592-b89a-35778fbe3045 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:27.025261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b75392-a291-4592-b89a-35778fbe3045 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:27.025529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b75392-a291-4592-b89a-35778fbe3045 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:27.026484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b75392-a291-4592-b89a-35778fbe3045 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:27.026594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b75392-a291-4592-b89a-35778fbe3045 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:27.028036 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b75392-a291-4592-b89a-35778fbe3045 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:27.048395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 427/841] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 9 13:02:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:27.053713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b75392-a291-4592-b89a-35778fbe3045 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:27.088526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 415/842] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:27.151355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:27.167979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b75392-a291-4592-b89a-35778fbe3045 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:27.207394 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e6956-7ae7-4170-b205-71b8ad54afa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:27.208446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e6956-7ae7-4170-b205-71b8ad54afa3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:27.208927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e6956-7ae7-4170-b205-71b8ad54afa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:27.209397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e6956-7ae7-4170-b205-71b8ad54afa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:27.278568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e6956-7ae7-4170-b205-71b8ad54afa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:27.280413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e6956-7ae7-4170-b205-71b8ad54afa3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:27.370155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:27.386141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745c333-0627-40c4-b158-b465b97254ab None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:27.386896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 428/843] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 13:02:27.392788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e6956-7ae7-4170-b205-71b8ad54afa3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:27.496040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e6956-7ae7-4170-b205-71b8ad54afa3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:27.500692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745c333-0627-40c4-b158-b465b97254ab None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:27.534740 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326103f-22d5-478d-bd02-73f752160d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:27.535431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326103f-22d5-478d-bd02-73f752160d7d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:27.535762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326103f-22d5-478d-bd02-73f752160d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:27.536032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326103f-22d5-478d-bd02-73f752160d7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:27.548298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326103f-22d5-478d-bd02-73f752160d7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:27.549764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326103f-22d5-478d-bd02-73f752160d7d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-4326103f-22d5-478d-bd02-73f752160d7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:27.567542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:27.568552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:27.569480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:27.569480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:27.569480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:27.573507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 429/844] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:27.574690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326103f-22d5-478d-bd02-73f752160d7d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:27.620989 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:27.665522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:27.675669 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b343b-b70d-4340-bdcd-bb5800ca0618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:27.676696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b343b-b70d-4340-bdcd-bb5800ca0618 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:27.677186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b343b-b70d-4340-bdcd-bb5800ca0618 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:27.677701 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b343b-b70d-4340-bdcd-bb5800ca0618 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:27.678480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b343b-b70d-4340-bdcd-bb5800ca0618 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:27.680287 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b343b-b70d-4340-bdcd-bb5800ca0618 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:27.686918 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326103f-22d5-478d-bd02-73f752160d7d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ba0b343b-b70d-4340-bdcd-bb5800ca0618 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:27.690874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:27.691975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 13:02:27.692840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:27.699164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 416/845] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:02:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:27.701712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b343b-b70d-4340-bdcd-bb5800ca0618 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:27.722963 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b28f0-6b34-4a99-a304-ec5e5f0108d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:27.723755 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b28f0-6b34-4a99-a304-ec5e5f0108d9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:27.723877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b28f0-6b34-4a99-a304-ec5e5f0108d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:27.724125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b28f0-6b34-4a99-a304-ec5e5f0108d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:27.724761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b28f0-6b34-4a99-a304-ec5e5f0108d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:27.725058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b28f0-6b34-4a99-a304-ec5e5f0108d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:27.726417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b28f0-6b34-4a99-a304-ec5e5f0108d9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:27.745274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 430/846] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:27.746347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b28f0-6b34-4a99-a304-ec5e5f0108d9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:27.814810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b343b-b70d-4340-bdcd-bb5800ca0618 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:27.850260 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:27.854456 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b28f0-6b34-4a99-a304-ec5e5f0108d9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:27.855218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecca213-71a1-4191-b875-8104f93f5d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:27.856066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecca213-71a1-4191-b875-8104f93f5d09 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:27.857031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecca213-71a1-4191-b875-8104f93f5d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:27.857599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecca213-71a1-4191-b875-8104f93f5d09 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:27.859175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecca213-71a1-4191-b875-8104f93f5d09 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:27.859713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecca213-71a1-4191-b875-8104f93f5d09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:27.861290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecca213-71a1-4191-b875-8104f93f5d09 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:27.871356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 417/847] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:02:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:27.872793 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecca213-71a1-4191-b875-8104f93f5d09 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:27.894494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da86b2c-aadf-4282-ab51-0ebc417391d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:27.895156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da86b2c-aadf-4282-ab51-0ebc417391d1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:27.895473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da86b2c-aadf-4282-ab51-0ebc417391d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:27.895783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da86b2c-aadf-4282-ab51-0ebc417391d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:27.902800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da86b2c-aadf-4282-ab51-0ebc417391d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:27.904254 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da86b2c-aadf-4282-ab51-0ebc417391d1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-2da86b2c-aadf-4282-ab51-0ebc417391d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:27.918048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:27.918997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:27.920316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:27.920316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:27.920316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:27.920316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:27.920316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:27.920316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:27.920316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:27.920316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:27.924889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 431/848] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:27.925911 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da86b2c-aadf-4282-ab51-0ebc417391d1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:27.940068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:27.990584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecca213-71a1-4191-b875-8104f93f5d09 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:28.029834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f53ee-9dc5-4eae-9c23-e3bccd8d39ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:28.031122 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f53ee-9dc5-4eae-9c23-e3bccd8d39ca None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:28.031609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f53ee-9dc5-4eae-9c23-e3bccd8d39ca None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:28.032081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f53ee-9dc5-4eae-9c23-e3bccd8d39ca None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:28.032935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f53ee-9dc5-4eae-9c23-e3bccd8d39ca None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:28.033274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da86b2c-aadf-4282-ab51-0ebc417391d1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:28.033668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f53ee-9dc5-4eae-9c23-e3bccd8d39ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:28.035365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f53ee-9dc5-4eae-9c23-e3bccd8d39ca None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:28.047196 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 418/849] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:02:27 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:28.071725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92138ec8-fcee-477b-a8be-8333b02ce6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:28.072753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92138ec8-fcee-477b-a8be-8333b02ce6ae None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:28.073238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92138ec8-fcee-477b-a8be-8333b02ce6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:28.073712 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92138ec8-fcee-477b-a8be-8333b02ce6ae None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:28.074641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92138ec8-fcee-477b-a8be-8333b02ce6ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:28.075218 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92138ec8-fcee-477b-a8be-8333b02ce6ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:28.077470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92138ec8-fcee-477b-a8be-8333b02ce6ae None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:28.092490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 432/850] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:28.098376 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f53ee-9dc5-4eae-9c23-e3bccd8d39ca None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:28.216208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f53ee-9dc5-4eae-9c23-e3bccd8d39ca None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:28.240039 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:28.256616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2b7ab-55e5-4eb3-8b59-72a61a357ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:28.257554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2b7ab-55e5-4eb3-8b59-72a61a357ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:28.258036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2b7ab-55e5-4eb3-8b59-72a61a357ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:28.258528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2b7ab-55e5-4eb3-8b59-72a61a357ab4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:28.259508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2b7ab-55e5-4eb3-8b59-72a61a357ab4 None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=c361a3e83f8e45c2b4d28f97f0a9d069) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:28.260019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2b7ab-55e5-4eb3-8b59-72a61a357ab4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:28.261727 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2b7ab-55e5-4eb3-8b59-72a61a357ab4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:28.268902 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 419/851] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:28 2022] GET /identity/v3/role_assignments?user.id=c361a3e83f8e45c2b4d28f97f0a9d069&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:28.409964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc093-c73f-4cb5-9f31-23700653dd96 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:28.410163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc093-c73f-4cb5-9f31-23700653dd96 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:28.410426 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc093-c73f-4cb5-9f31-23700653dd96 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:28.412221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 433/852] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:28.418284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5680d1-b5ec-40b2-aa6f-4a51b3cb6496 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:28.418579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5680d1-b5ec-40b2-aa6f-4a51b3cb6496 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:28.418836 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5680d1-b5ec-40b2-aa6f-4a51b3cb6496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:28.439178 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ef5680d1-b5ec-40b2-aa6f-4a51b3cb6496 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:28.440469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ef5680d1-b5ec-40b2-aa6f-4a51b3cb6496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:28.496442 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52702ba4-30e4-4b32-8343-c065fb84e7e1 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:28.496783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52702ba4-30e4-4b32-8343-c065fb84e7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:28.496997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52702ba4-30e4-4b32-8343-c065fb84e7e1 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:28.498658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 434/853] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:28.504260 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec049dc-04f3-4bd6-a4aa-08e3f1d5e9e9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:28.504550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec049dc-04f3-4bd6-a4aa-08e3f1d5e9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:28.504808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec049dc-04f3-4bd6-a4aa-08e3f1d5e9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:28.517896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 420/854] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:28.528324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5ec049dc-04f3-4bd6-a4aa-08e3f1d5e9e9 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:28.529847 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5ec049dc-04f3-4bd6-a4aa-08e3f1d5e9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:28.547441 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:28.605169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 435/855] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:28.787819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5680d1-b5ec-40b2-aa6f-4a51b3cb6496 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:28.886141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec049dc-04f3-4bd6-a4aa-08e3f1d5e9e9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:28.895166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5680d1-b5ec-40b2-aa6f-4a51b3cb6496 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:28.911517 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:28.933400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52a527-28a0-4799-8514-dfd823bae9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:28.934397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52a527-28a0-4799-8514-dfd823bae9dc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:28.934898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52a527-28a0-4799-8514-dfd823bae9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:28.935648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52a527-28a0-4799-8514-dfd823bae9dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:28.946717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52a527-28a0-4799-8514-dfd823bae9dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9d52a527-28a0-4799-8514-dfd823bae9dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:28.951748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:28.952890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:28.953723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:28.953723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:28.953723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:28.953723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:28.953723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:28.953723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:28.953723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:28.953723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:28.953723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:28.953723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:28.953723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:02:28.953723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:28.960853 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 421/856] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:02:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:28.962362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52a527-28a0-4799-8514-dfd823bae9dc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:29.001459 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec049dc-04f3-4bd6-a4aa-08e3f1d5e9e9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:29.039114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a62eb0-0c82-4e17-8330-78278630162e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:29.039645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a62eb0-0c82-4e17-8330-78278630162e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:29.039965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a62eb0-0c82-4e17-8330-78278630162e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:29.040236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a62eb0-0c82-4e17-8330-78278630162e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:29.040879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a62eb0-0c82-4e17-8330-78278630162e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:29.042335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a62eb0-0c82-4e17-8330-78278630162e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-73a62eb0-0c82-4e17-8330-78278630162e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:29.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:29.060044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 13:02:29.061766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:29.064473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 436/857] 10.222.0.17 () {56 vars in 1132 bytes} [Fri Sep 9 13:02:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:29.070890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a62eb0-0c82-4e17-8330-78278630162e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:29.079342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52a527-28a0-4799-8514-dfd823bae9dc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:29.086392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:29.117408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3807d-9847-4b3f-8b6f-b2ccc1e26472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:29.118377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3807d-9847-4b3f-8b6f-b2ccc1e26472 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:29.118856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3807d-9847-4b3f-8b6f-b2ccc1e26472 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:29.119381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3807d-9847-4b3f-8b6f-b2ccc1e26472 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:29.120244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3807d-9847-4b3f-8b6f-b2ccc1e26472 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:29.120745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3807d-9847-4b3f-8b6f-b2ccc1e26472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:29.122325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3807d-9847-4b3f-8b6f-b2ccc1e26472 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:29.136093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 422/858] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:02:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:29.140799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3807d-9847-4b3f-8b6f-b2ccc1e26472 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:29.176170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:29.188940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a62eb0-0c82-4e17-8330-78278630162e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:29.228064 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687fdb0-1854-4256-85c8-abb5677804b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:29.228964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687fdb0-1854-4256-85c8-abb5677804b5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:29.229337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687fdb0-1854-4256-85c8-abb5677804b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:29.229734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687fdb0-1854-4256-85c8-abb5677804b5 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:29.230785 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687fdb0-1854-4256-85c8-abb5677804b5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:29.231330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687fdb0-1854-4256-85c8-abb5677804b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:29.233353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687fdb0-1854-4256-85c8-abb5677804b5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:29.243693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 437/859] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 13:02:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:29.244611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687fdb0-1854-4256-85c8-abb5677804b5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:29.250054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3807d-9847-4b3f-8b6f-b2ccc1e26472 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:29.290926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb2ba6-74c8-4322-9a0f-b5b5c4a5cd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:29.291806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb2ba6-74c8-4322-9a0f-b5b5c4a5cd44 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:29.292251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb2ba6-74c8-4322-9a0f-b5b5c4a5cd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:29.292682 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb2ba6-74c8-4322-9a0f-b5b5c4a5cd44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:29.305053 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb2ba6-74c8-4322-9a0f-b5b5c4a5cd44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:29.306993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb2ba6-74c8-4322-9a0f-b5b5c4a5cd44 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-dfcb2ba6-74c8-4322-9a0f-b5b5c4a5cd44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:29.324131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:29.325386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:29.326494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:29.326494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:29.326494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:29.333777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 423/860] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:29.335258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb2ba6-74c8-4322-9a0f-b5b5c4a5cd44 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:29.340472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:29.369459 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687fdb0-1854-4256-85c8-abb5677804b5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:29.410475 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac213e-08ee-4676-80a3-ce8f1b1999c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:29.411103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac213e-08ee-4676-80a3-ce8f1b1999c6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:29.411382 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac213e-08ee-4676-80a3-ce8f1b1999c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:29.411727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac213e-08ee-4676-80a3-ce8f1b1999c6 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:29.412275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac213e-08ee-4676-80a3-ce8f1b1999c6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:29.412573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac213e-08ee-4676-80a3-ce8f1b1999c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:29.413925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac213e-08ee-4676-80a3-ce8f1b1999c6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:29.424954 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 438/861] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 13:02:29 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:29.434247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:29.457113 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb2ba6-74c8-4322-9a0f-b5b5c4a5cd44 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:29.494466 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680ac88-8a8e-4aa4-8e16-6a93ec028c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:29.495162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680ac88-8a8e-4aa4-8e16-6a93ec028c39 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:29.495443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680ac88-8a8e-4aa4-8e16-6a93ec028c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:29.495715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680ac88-8a8e-4aa4-8e16-6a93ec028c39 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:29.496375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680ac88-8a8e-4aa4-8e16-6a93ec028c39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:29.496679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680ac88-8a8e-4aa4-8e16-6a93ec028c39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:29.498078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680ac88-8a8e-4aa4-8e16-6a93ec028c39 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:29.512406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 424/862] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:29.518188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac213e-08ee-4676-80a3-ce8f1b1999c6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:29.631603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac213e-08ee-4676-80a3-ce8f1b1999c6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:29.668234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f58b8f-af86-41c1-91cd-4f8a11948c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:29.668865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f58b8f-af86-41c1-91cd-4f8a11948c30 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:29.669140 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f58b8f-af86-41c1-91cd-4f8a11948c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:29.669410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f58b8f-af86-41c1-91cd-4f8a11948c30 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:29.677808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f58b8f-af86-41c1-91cd-4f8a11948c30 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:29.679270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f58b8f-af86-41c1-91cd-4f8a11948c30 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:29.682478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-22f58b8f-af86-41c1-91cd-4f8a11948c30 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:29.693182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:29.694181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:29.695157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:29.695157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:29.695157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:29.695157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:29.695157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:29.695157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:29.695157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:29.695157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:29.695157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 13:02:29.695157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:29.695157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 439/863] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 13:02:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:29.700241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680ac88-8a8e-4aa4-8e16-6a93ec028c39 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:29.803595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680ac88-8a8e-4aa4-8e16-6a93ec028c39 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:29.845023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb06b98-7bcf-4646-9e9e-08a17e0ae919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:29.845642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb06b98-7bcf-4646-9e9e-08a17e0ae919 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:29.845914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb06b98-7bcf-4646-9e9e-08a17e0ae919 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:29.846183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb06b98-7bcf-4646-9e9e-08a17e0ae919 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:29.846904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb06b98-7bcf-4646-9e9e-08a17e0ae919 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:29.847232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb06b98-7bcf-4646-9e9e-08a17e0ae919 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:29.848653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb06b98-7bcf-4646-9e9e-08a17e0ae919 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:29.867302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 425/864] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 9 13:02:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:29.876151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f58b8f-af86-41c1-91cd-4f8a11948c30 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:29.983719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f58b8f-af86-41c1-91cd-4f8a11948c30 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:30.021731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6670264-745a-4239-9b11-0abd355570c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:30.022348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6670264-745a-4239-9b11-0abd355570c8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:30.022628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6670264-745a-4239-9b11-0abd355570c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:30.022905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6670264-745a-4239-9b11-0abd355570c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:30.093158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:30.096129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6670264-745a-4239-9b11-0abd355570c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:30.097741 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6670264-745a-4239-9b11-0abd355570c8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:30.202841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 440/865] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:29 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:02:30.215170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb06b98-7bcf-4646-9e9e-08a17e0ae919 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:30.224070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:30.328734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb06b98-7bcf-4646-9e9e-08a17e0ae919 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:30.364429 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b05e91-d9c7-4533-9983-c396c93ec13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:30.365306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b05e91-d9c7-4533-9983-c396c93ec13b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:30.365733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b05e91-d9c7-4533-9983-c396c93ec13b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:30.366175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b05e91-d9c7-4533-9983-c396c93ec13b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:30.377375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b05e91-d9c7-4533-9983-c396c93ec13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:30.379016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b05e91-d9c7-4533-9983-c396c93ec13b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-58b05e91-d9c7-4533-9983-c396c93ec13b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:30.394717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:30.395719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:30.396613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:30.396613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:30.396613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:30.396613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 426/866] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:30.401602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6670264-745a-4239-9b11-0abd355570c8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:30.412902 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52d643-6577-4c7f-bd08-e48ff05a808c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:30.413480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52d643-6577-4c7f-bd08-e48ff05a808c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:30.414299 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52d643-6577-4c7f-bd08-e48ff05a808c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:30.416888 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 427/867] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:02:30.423025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27a857-f525-49c7-b016-af56090914c8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:30.423673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27a857-f525-49c7-b016-af56090914c8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:30.424125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27a857-f525-49c7-b016-af56090914c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:30.450996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1a27a857-f525-49c7-b016-af56090914c8 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:30.452988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1a27a857-f525-49c7-b016-af56090914c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:30.510984 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6670264-745a-4239-9b11-0abd355570c8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:30.515436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:30.524153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:30.547091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 428/868] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:30.562685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4176bfe-b646-48c2-ab0a-375b16097d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:30.563341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4176bfe-b646-48c2-ab0a-375b16097d77 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:30.563627 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4176bfe-b646-48c2-ab0a-375b16097d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:30.563896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4176bfe-b646-48c2-ab0a-375b16097d77 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:30.564538 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4176bfe-b646-48c2-ab0a-375b16097d77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:30.564843 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4176bfe-b646-48c2-ab0a-375b16097d77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:30.566206 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4176bfe-b646-48c2-ab0a-375b16097d77 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:30.582247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 441/869] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:30.583675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27a857-f525-49c7-b016-af56090914c8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:30.637364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd690af2-7609-47cb-9066-8878af1824af None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:30.637877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd690af2-7609-47cb-9066-8878af1824af None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:30.637957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd690af2-7609-47cb-9066-8878af1824af None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:30.639599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 442/870] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:02:30.644505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e36067-68c7-4fa3-a195-cef3ced7b638 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:30.644792 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e36067-68c7-4fa3-a195-cef3ced7b638 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:30.645073 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e36067-68c7-4fa3-a195-cef3ced7b638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:30.665177 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-86e36067-68c7-4fa3-a195-cef3ced7b638 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:30.666374 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-86e36067-68c7-4fa3-a195-cef3ced7b638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:30.695459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27a857-f525-49c7-b016-af56090914c8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:30.733443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a9f0f-bfde-41db-85ca-e8b2d7b0a6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:30.733986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a9f0f-bfde-41db-85ca-e8b2d7b0a6ae None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:30.734261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a9f0f-bfde-41db-85ca-e8b2d7b0a6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:30.734589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a9f0f-bfde-41db-85ca-e8b2d7b0a6ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:30.742200 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a9f0f-bfde-41db-85ca-e8b2d7b0a6ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:30.743782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a9f0f-bfde-41db-85ca-e8b2d7b0a6ae None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:30.750261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 443/871] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-424a9f0f-bfde-41db-85ca-e8b2d7b0a6ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:30.764717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:30.765803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:30.766746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:30.766746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:30.766746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:30.766746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:30.766746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:30.766746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:30.766746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:30.766746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:30.773112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 429/872] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:30.774614 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e36067-68c7-4fa3-a195-cef3ced7b638 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:30.860772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a9f0f-bfde-41db-85ca-e8b2d7b0a6ae None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:30.870462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:30.883193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e36067-68c7-4fa3-a195-cef3ced7b638 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:30.921047 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:30.924742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173eddfe-44e4-4736-8bd6-905400d2a1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:30.925360 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173eddfe-44e4-4736-8bd6-905400d2a1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:30.925640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173eddfe-44e4-4736-8bd6-905400d2a1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:30.925913 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173eddfe-44e4-4736-8bd6-905400d2a1a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:30.926602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173eddfe-44e4-4736-8bd6-905400d2a1a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:30.926910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173eddfe-44e4-4736-8bd6-905400d2a1a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:30.929216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173eddfe-44e4-4736-8bd6-905400d2a1a8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:30.949236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 444/873] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:30.952061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173eddfe-44e4-4736-8bd6-905400d2a1a8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:30.992147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a9f0f-bfde-41db-85ca-e8b2d7b0a6ae None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:31.032845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15fffb-80d9-45bc-b236-def27f7c0b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:31.033481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15fffb-80d9-45bc-b236-def27f7c0b55 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:31.033771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15fffb-80d9-45bc-b236-def27f7c0b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:31.034044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15fffb-80d9-45bc-b236-def27f7c0b55 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:31.034704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15fffb-80d9-45bc-b236-def27f7c0b55 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:31.036103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15fffb-80d9-45bc-b236-def27f7c0b55 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:31.053141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 430/874] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:02:30 2022] POST /identity/v3/services => generated 251 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:02:31.061751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15fffb-80d9-45bc-b236-def27f7c0b55 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:31.071853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173eddfe-44e4-4736-8bd6-905400d2a1a8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:31.121186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3991d7-cbe3-4b3a-a0ec-6b4f9b9eafda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:31.121811 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3991d7-cbe3-4b3a-a0ec-6b4f9b9eafda None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:31.122090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3991d7-cbe3-4b3a-a0ec-6b4f9b9eafda None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:31.122425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3991d7-cbe3-4b3a-a0ec-6b4f9b9eafda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:31.123289 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3991d7-cbe3-4b3a-a0ec-6b4f9b9eafda None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=8d343a3db21c47e1a85a739a55559c97) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:31.123607 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3991d7-cbe3-4b3a-a0ec-6b4f9b9eafda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:31.125186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3991d7-cbe3-4b3a-a0ec-6b4f9b9eafda None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:31.138584 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 445/875] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:30 2022] GET /identity/v3/role_assignments?user.id=8d343a3db21c47e1a85a739a55559c97&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:31.214908 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15fffb-80d9-45bc-b236-def27f7c0b55 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:31.227614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:31.263580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1c662-2a80-4f91-babe-ffd640d9187a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:31.264656 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1c662-2a80-4f91-babe-ffd640d9187a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:31.265194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1c662-2a80-4f91-babe-ffd640d9187a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:31.265691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1c662-2a80-4f91-babe-ffd640d9187a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:31.266503 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1c662-2a80-4f91-babe-ffd640d9187a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:31.268165 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1c662-2a80-4f91-babe-ffd640d9187a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c5b1c662-2a80-4f91-babe-ffd640d9187a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:31.278443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:31.280274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:31.281292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:31.283898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 431/876] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 13:02:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:31.290070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3991d7-cbe3-4b3a-a0ec-6b4f9b9eafda None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:31.305933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:31.406885 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3991d7-cbe3-4b3a-a0ec-6b4f9b9eafda None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:31.445994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafc517-0270-4a05-87bc-d11c89026f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:31.446613 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafc517-0270-4a05-87bc-d11c89026f13 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:31.446895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafc517-0270-4a05-87bc-d11c89026f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:31.447204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafc517-0270-4a05-87bc-d11c89026f13 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:31.447844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafc517-0270-4a05-87bc-d11c89026f13 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:31.448149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafc517-0270-4a05-87bc-d11c89026f13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:31.449519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafc517-0270-4a05-87bc-d11c89026f13 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:31.455023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 446/877] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 13:02:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:31.460295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1c662-2a80-4f91-babe-ffd640d9187a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:31.572007 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1c662-2a80-4f91-babe-ffd640d9187a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:31.626422 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcc35b-689d-4402-817e-118a49949324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:31.627070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcc35b-689d-4402-817e-118a49949324 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:31.627362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcc35b-689d-4402-817e-118a49949324 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:31.627637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcc35b-689d-4402-817e-118a49949324 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:31.628281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcc35b-689d-4402-817e-118a49949324 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:31.628587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcc35b-689d-4402-817e-118a49949324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:31.629979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcc35b-689d-4402-817e-118a49949324 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:31.635382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 432/878] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 13:02:31 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:31.831815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96b1f9-1bda-4e69-9eea-61478aedb515 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:31.832902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96b1f9-1bda-4e69-9eea-61478aedb515 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:31.833406 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96b1f9-1bda-4e69-9eea-61478aedb515 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:31.835282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 447/879] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:31.840941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa65eb-784c-41da-8279-018b35243482 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:31.841232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa65eb-784c-41da-8279-018b35243482 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:31.841509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa65eb-784c-41da-8279-018b35243482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:31.862636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e5aa65eb-784c-41da-8279-018b35243482 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:31.864224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e5aa65eb-784c-41da-8279-018b35243482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:31.896135 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:31.943173 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 433/880] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:31 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:32.214754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96b1f9-1bda-4e69-9eea-61478aedb515 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:32.295850 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:32.323021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96b1f9-1bda-4e69-9eea-61478aedb515 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:32.360094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc6b8a-94ce-4bab-b684-2655a89ba413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:32.360724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc6b8a-94ce-4bab-b684-2655a89ba413 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:32.361059 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc6b8a-94ce-4bab-b684-2655a89ba413 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:32.361340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc6b8a-94ce-4bab-b684-2655a89ba413 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:32.362009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc6b8a-94ce-4bab-b684-2655a89ba413 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:32.363402 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc6b8a-94ce-4bab-b684-2655a89ba413 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:32.387242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 448/881] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:02:32 2022] POST /identity/v3/services => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:33.361479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea6df3-3643-4164-a3ec-36b1215c5795 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:33.362050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea6df3-3643-4164-a3ec-36b1215c5795 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:33.362050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea6df3-3643-4164-a3ec-36b1215c5795 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:33.364044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 434/882] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:33.370026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811221c-60a4-4265-a666-d4660e561afd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:33.370533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811221c-60a4-4265-a666-d4660e561afd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:33.370735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811221c-60a4-4265-a666-d4660e561afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:33.390833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5811221c-60a4-4265-a666-d4660e561afd None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:33.392068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5811221c-60a4-4265-a666-d4660e561afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:33.467951 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 449/883] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:33.469731 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c2626-213c-4f7f-92d0-3cd21ef2f219 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:33.470044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c2626-213c-4f7f-92d0-3cd21ef2f219 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:33.470318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c2626-213c-4f7f-92d0-3cd21ef2f219 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:33.477528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 435/884] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:33.479073 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f07741-63e9-4450-9bb6-b52b31d683ba None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:33.483075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f07741-63e9-4450-9bb6-b52b31d683ba None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:33.483075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f07741-63e9-4450-9bb6-b52b31d683ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:33.498905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-13f07741-63e9-4450-9bb6-b52b31d683ba None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:33.500120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-13f07741-63e9-4450-9bb6-b52b31d683ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:33.563556 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:33.574449 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 450/885] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:33.759015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c2626-213c-4f7f-92d0-3cd21ef2f219 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:33.850427 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f07741-63e9-4450-9bb6-b52b31d683ba None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:33.878046 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c2626-213c-4f7f-92d0-3cd21ef2f219 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:33.916720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b4cad-60eb-44ea-9ff2-76065b40ade4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:33.917345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b4cad-60eb-44ea-9ff2-76065b40ade4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:33.917626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b4cad-60eb-44ea-9ff2-76065b40ade4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:33.917912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b4cad-60eb-44ea-9ff2-76065b40ade4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:33.918472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b4cad-60eb-44ea-9ff2-76065b40ade4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:33.920390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b4cad-60eb-44ea-9ff2-76065b40ade4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-612b4cad-60eb-44ea-9ff2-76065b40ade4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:33.929277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:33.930397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:33.931904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 436/886] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:02:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:33.932781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e59fa-33d1-4ed1-b734-e472f5f325a5 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:33.932781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e59fa-33d1-4ed1-b734-e472f5f325a5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:33.932781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e59fa-33d1-4ed1-b734-e472f5f325a5 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:33.933885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 437/887] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:33.939989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85462e0-a84d-4f6e-9f2a-96bedd4c54ca None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:33.940274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85462e0-a84d-4f6e-9f2a-96bedd4c54ca None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:33.940550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85462e0-a84d-4f6e-9f2a-96bedd4c54ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:33.960530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e85462e0-a84d-4f6e-9f2a-96bedd4c54ca None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:33.961718 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e85462e0-a84d-4f6e-9f2a-96bedd4c54ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:33.965636 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f07741-63e9-4450-9bb6-b52b31d683ba None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:33.986545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:34.002185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3df13-993f-4744-a543-0688fc6e85c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:34.002824 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3df13-993f-4744-a543-0688fc6e85c0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:34.003156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3df13-993f-4744-a543-0688fc6e85c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:34.003436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3df13-993f-4744-a543-0688fc6e85c0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:34.014420 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b3df13-993f-4744-a543-0688fc6e85c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-b9b3df13-993f-4744-a543-0688fc6e85c0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:34.019336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:34.020361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 451/888] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 9 13:02:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:34.021242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3df13-993f-4744-a543-0688fc6e85c0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:34.038868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 438/889] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:34.040068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85462e0-a84d-4f6e-9f2a-96bedd4c54ca None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:34.089485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:34.148721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3df13-993f-4744-a543-0688fc6e85c0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:34.173734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85462e0-a84d-4f6e-9f2a-96bedd4c54ca None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:34.187952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f3389-8a63-436e-a4d9-e13ba43bd479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:34.188960 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f3389-8a63-436e-a4d9-e13ba43bd479 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:34.189332 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f3389-8a63-436e-a4d9-e13ba43bd479 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:34.189708 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f3389-8a63-436e-a4d9-e13ba43bd479 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:34.190448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f3389-8a63-436e-a4d9-e13ba43bd479 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:34.190856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f3389-8a63-436e-a4d9-e13ba43bd479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:34.192593 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f3389-8a63-436e-a4d9-e13ba43bd479 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:34.202744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 452/890] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:02:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:34.203809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f3389-8a63-436e-a4d9-e13ba43bd479 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:34.220183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5a6cd-3813-4696-aeb9-79970bf40061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:34.220859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5a6cd-3813-4696-aeb9-79970bf40061 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:34.221141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5a6cd-3813-4696-aeb9-79970bf40061 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:34.221413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5a6cd-3813-4696-aeb9-79970bf40061 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:34.222132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5a6cd-3813-4696-aeb9-79970bf40061 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:34.222436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5a6cd-3813-4696-aeb9-79970bf40061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:34.224123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5a6cd-3813-4696-aeb9-79970bf40061 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:34.237751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 439/891] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 13:02:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:34.238782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5a6cd-3813-4696-aeb9-79970bf40061 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:34.265106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:34.332202 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f3389-8a63-436e-a4d9-e13ba43bd479 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:34.347370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:34.354603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5a6cd-3813-4696-aeb9-79970bf40061 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:34.368872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbba5e-ac9b-4cd7-8cec-bbceb26d3346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:34.369522 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbba5e-ac9b-4cd7-8cec-bbceb26d3346 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:34.369801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbba5e-ac9b-4cd7-8cec-bbceb26d3346 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:34.370067 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbba5e-ac9b-4cd7-8cec-bbceb26d3346 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:34.370739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbba5e-ac9b-4cd7-8cec-bbceb26d3346 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:34.370992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbba5e-ac9b-4cd7-8cec-bbceb26d3346 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:34.372375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbba5e-ac9b-4cd7-8cec-bbceb26d3346 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:34.382558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 453/892] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:02:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:34.382558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbba5e-ac9b-4cd7-8cec-bbceb26d3346 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:34.396056 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc786de-087d-4a23-9bd9-2acc14086ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:34.396700 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc786de-087d-4a23-9bd9-2acc14086ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:34.396978 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc786de-087d-4a23-9bd9-2acc14086ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:34.397251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc786de-087d-4a23-9bd9-2acc14086ee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:34.408759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc786de-087d-4a23-9bd9-2acc14086ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:34.410201 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc786de-087d-4a23-9bd9-2acc14086ee8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9dc786de-087d-4a23-9bd9-2acc14086ee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:34.424431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:34.425456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:34.426392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:34.426392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:34.426392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:34.426392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 440/893] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:34.426392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc786de-087d-4a23-9bd9-2acc14086ee8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:34.490095 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbba5e-ac9b-4cd7-8cec-bbceb26d3346 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:34.503907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:34.527594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4121c7-19b2-46de-ba7f-4ff01d155071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:34.528136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4121c7-19b2-46de-ba7f-4ff01d155071 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:34.528416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4121c7-19b2-46de-ba7f-4ff01d155071 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:34.528684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4121c7-19b2-46de-ba7f-4ff01d155071 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:34.529394 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4121c7-19b2-46de-ba7f-4ff01d155071 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:34.530861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc786de-087d-4a23-9bd9-2acc14086ee8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:34.531087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4121c7-19b2-46de-ba7f-4ff01d155071 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:34.551107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 454/894] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:02:34 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:34.551107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4121c7-19b2-46de-ba7f-4ff01d155071 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:34.588160 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4dcfe-db64-451e-ab7f-c3ef9d1c9f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:34.589142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4dcfe-db64-451e-ab7f-c3ef9d1c9f18 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:34.589630 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4dcfe-db64-451e-ab7f-c3ef9d1c9f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:34.590078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4dcfe-db64-451e-ab7f-c3ef9d1c9f18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:34.591129 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4dcfe-db64-451e-ab7f-c3ef9d1c9f18 None admin] RBAC: Adding query filter params (interface=public, service_id=e31c16ef13344f8887a112e7ee6feb6d, region_id=RegionOne) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:34.592367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4dcfe-db64-451e-ab7f-c3ef9d1c9f18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:34.593883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4dcfe-db64-451e-ab7f-c3ef9d1c9f18 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:34.603091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 441/895] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 13:02:34 2022] GET /identity/v3/endpoints?service_id=e31c16ef13344f8887a112e7ee6feb6d&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:34.609967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:34.614040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4dcfe-db64-451e-ab7f-c3ef9d1c9f18 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:34.685650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4121c7-19b2-46de-ba7f-4ff01d155071 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:34.725391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2873d5c-c349-407b-9d62-f2f30f149df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:34.726015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2873d5c-c349-407b-9d62-f2f30f149df7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:34.726292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2873d5c-c349-407b-9d62-f2f30f149df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:34.726564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2873d5c-c349-407b-9d62-f2f30f149df7 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:34.727424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2873d5c-c349-407b-9d62-f2f30f149df7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:34.727733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2873d5c-c349-407b-9d62-f2f30f149df7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:34.729550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2873d5c-c349-407b-9d62-f2f30f149df7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:34.735118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4dcfe-db64-451e-ab7f-c3ef9d1c9f18 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:34.743537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 455/896] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:34.744835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fbfe9-1e0c-431b-a9b2-acf9c3c4d4bc None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:34.745115 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fbfe9-1e0c-431b-a9b2-acf9c3c4d4bc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:34.745380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fbfe9-1e0c-431b-a9b2-acf9c3c4d4bc None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:34.746796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 456/897] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:34.749178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fbfe9-1e0c-431b-a9b2-acf9c3c4d4bc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:34.781808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ed4f7-6935-4c17-9dad-8a532e050094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:34.782354 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ed4f7-6935-4c17-9dad-8a532e050094 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:34.782628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ed4f7-6935-4c17-9dad-8a532e050094 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:34.782894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ed4f7-6935-4c17-9dad-8a532e050094 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:34.783578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ed4f7-6935-4c17-9dad-8a532e050094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:34.785009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ed4f7-6935-4c17-9dad-8a532e050094 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:34.786142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:34.791788 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 442/898] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 13:02:34 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:02:34.793025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba1264-1265-4df4-9f4a-94d2a1a146ec None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:34.793306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba1264-1265-4df4-9f4a-94d2a1a146ec None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:34.793881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba1264-1265-4df4-9f4a-94d2a1a146ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:34.815222 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5bba1264-1265-4df4-9f4a-94d2a1a146ec None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:34.817171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5bba1264-1265-4df4-9f4a-94d2a1a146ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:34.874864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:34.881974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fbfe9-1e0c-431b-a9b2-acf9c3c4d4bc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:34.898650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 443/899] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:34.923198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b15734-98fc-43ce-b104-ed6c76905acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:34.923916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b15734-98fc-43ce-b104-ed6c76905acd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:34.924201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b15734-98fc-43ce-b104-ed6c76905acd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:34.924476 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b15734-98fc-43ce-b104-ed6c76905acd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:34.933449 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b15734-98fc-43ce-b104-ed6c76905acd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:34.936080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b15734-98fc-43ce-b104-ed6c76905acd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-15b15734-98fc-43ce-b104-ed6c76905acd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:34.949437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:34.950473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:34.951572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:34.951572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:34.951572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:34.951572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:34.951572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:34.951572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:34.951572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:34.951572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:34.951572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 13:02:34.951572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:34.951572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 457/900] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 13:02:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:34.959436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba1264-1265-4df4-9f4a-94d2a1a146ec None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:35.072179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba1264-1265-4df4-9f4a-94d2a1a146ec None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:35.100820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:35.110031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41872cb-a40c-489d-a21b-85ce2e872367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:35.110654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41872cb-a40c-489d-a21b-85ce2e872367 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:35.110931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41872cb-a40c-489d-a21b-85ce2e872367 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:35.111304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41872cb-a40c-489d-a21b-85ce2e872367 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:35.112049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41872cb-a40c-489d-a21b-85ce2e872367 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:35.112356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41872cb-a40c-489d-a21b-85ce2e872367 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:35.113855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41872cb-a40c-489d-a21b-85ce2e872367 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:35.133045 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 444/901] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 9 13:02:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:35.139737 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b15734-98fc-43ce-b104-ed6c76905acd None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:35.183067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41872cb-a40c-489d-a21b-85ce2e872367 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:35.254898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b15734-98fc-43ce-b104-ed6c76905acd None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:35.301629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0b1bb-9b27-471a-8081-f50918e1c86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:35.302246 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0b1bb-9b27-471a-8081-f50918e1c86c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:35.302524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0b1bb-9b27-471a-8081-f50918e1c86c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:35.302796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0b1bb-9b27-471a-8081-f50918e1c86c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:35.308367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41872cb-a40c-489d-a21b-85ce2e872367 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:35.332725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:35.349529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4ec41-c9cf-48bf-8ecb-721d8f5d5594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:35.350254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4ec41-c9cf-48bf-8ecb-721d8f5d5594 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:35.350615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4ec41-c9cf-48bf-8ecb-721d8f5d5594 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:35.350949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4ec41-c9cf-48bf-8ecb-721d8f5d5594 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:35.351676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4ec41-c9cf-48bf-8ecb-721d8f5d5594 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:35.353036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4ec41-c9cf-48bf-8ecb-721d8f5d5594 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d9b4ec41-c9cf-48bf-8ecb-721d8f5d5594 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:35.363484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:35.364505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:35.370633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 445/902] 10.222.0.17 () {56 vars in 1132 bytes} [Fri Sep 9 13:02:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:35.371485 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4ec41-c9cf-48bf-8ecb-721d8f5d5594 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:35.392605 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0b1bb-9b27-471a-8081-f50918e1c86c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:35.394261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0b1bb-9b27-471a-8081-f50918e1c86c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:35.439939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:35.509183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4ec41-c9cf-48bf-8ecb-721d8f5d5594 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:35.523169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 458/903] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:35 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:02:35.528377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0b1bb-9b27-471a-8081-f50918e1c86c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:35.550553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d9008-cfb5-45a1-ae97-f7cc2b1d76b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:35.551302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d9008-cfb5-45a1-ae97-f7cc2b1d76b2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:35.551838 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d9008-cfb5-45a1-ae97-f7cc2b1d76b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:35.551966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d9008-cfb5-45a1-ae97-f7cc2b1d76b2 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:35.552680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d9008-cfb5-45a1-ae97-f7cc2b1d76b2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:35.553058 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d9008-cfb5-45a1-ae97-f7cc2b1d76b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:35.554508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d9008-cfb5-45a1-ae97-f7cc2b1d76b2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:35.564765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 446/904] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 13:02:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:35.565829 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d9008-cfb5-45a1-ae97-f7cc2b1d76b2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:35.609000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:35.655536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0b1bb-9b27-471a-8081-f50918e1c86c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:35.692170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e012cd-ad97-4249-929e-e67d18c8cbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:35.693399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e012cd-ad97-4249-929e-e67d18c8cbff None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:35.693889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e012cd-ad97-4249-929e-e67d18c8cbff None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:35.694797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e012cd-ad97-4249-929e-e67d18c8cbff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:35.700444 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d9008-cfb5-45a1-ae97-f7cc2b1d76b2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:35.707385 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e012cd-ad97-4249-929e-e67d18c8cbff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:35.709555 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e012cd-ad97-4249-929e-e67d18c8cbff None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:35.714994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-05e012cd-ad97-4249-929e-e67d18c8cbff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:35.728495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:35.729519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:35.730397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:35.730397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:35.730397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:35.732636 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 459/905] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:35.739672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e012cd-ad97-4249-929e-e67d18c8cbff None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:35.745783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b9737-9c76-4255-b57a-c8b04a5cd817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:35.746817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b9737-9c76-4255-b57a-c8b04a5cd817 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:35.747370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b9737-9c76-4255-b57a-c8b04a5cd817 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:35.748470 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b9737-9c76-4255-b57a-c8b04a5cd817 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:35.749317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b9737-9c76-4255-b57a-c8b04a5cd817 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:35.749842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b9737-9c76-4255-b57a-c8b04a5cd817 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:35.752039 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b9737-9c76-4255-b57a-c8b04a5cd817 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:35.763283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 447/906] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 13:02:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:35.766128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b9737-9c76-4255-b57a-c8b04a5cd817 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:35.860956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e012cd-ad97-4249-929e-e67d18c8cbff None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:35.872435 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:35.894226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b9737-9c76-4255-b57a-c8b04a5cd817 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:35.904957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236308a-b642-4ea2-9ab1-db5eb0801b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:35.905671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236308a-b642-4ea2-9ab1-db5eb0801b7f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:35.906037 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236308a-b642-4ea2-9ab1-db5eb0801b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:35.906373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236308a-b642-4ea2-9ab1-db5eb0801b7f None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:35.907530 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236308a-b642-4ea2-9ab1-db5eb0801b7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:35.907901 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236308a-b642-4ea2-9ab1-db5eb0801b7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:35.915232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236308a-b642-4ea2-9ab1-db5eb0801b7f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:35.922428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 460/907] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:35.931166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236308a-b642-4ea2-9ab1-db5eb0801b7f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:35.950119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff81739-d727-4c37-9b94-78b54db02fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:35.951777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff81739-d727-4c37-9b94-78b54db02fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:35.952570 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff81739-d727-4c37-9b94-78b54db02fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:35.953058 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff81739-d727-4c37-9b94-78b54db02fb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:35.954260 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff81739-d727-4c37-9b94-78b54db02fb4 None admin] RBAC: Adding query filter params (interface=public, service_id=2df82dea41d34e79a5f1430d0f88a353, region_id=RegionOne) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:35.955138 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff81739-d727-4c37-9b94-78b54db02fb4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:35.956718 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff81739-d727-4c37-9b94-78b54db02fb4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:35.969542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 448/908] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 13:02:35 2022] GET /identity/v3/endpoints?service_id=2df82dea41d34e79a5f1430d0f88a353&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:35.970314 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff81739-d727-4c37-9b94-78b54db02fb4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:36.031441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:36.049343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236308a-b642-4ea2-9ab1-db5eb0801b7f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:36.088272 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3441f0-acd9-48f5-9a47-f140b9a66682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:36.088889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3441f0-acd9-48f5-9a47-f140b9a66682 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:36.089169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3441f0-acd9-48f5-9a47-f140b9a66682 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:36.089443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3441f0-acd9-48f5-9a47-f140b9a66682 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:36.098072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3441f0-acd9-48f5-9a47-f140b9a66682 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:36.099904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3441f0-acd9-48f5-9a47-f140b9a66682 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:36.100576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff81739-d727-4c37-9b94-78b54db02fb4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-9c3441f0-acd9-48f5-9a47-f140b9a66682 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:36.115561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:36.116506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:36.117401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:36.117401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:36.117401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:36.117401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:36.117401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:36.117401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:36.117401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:36.117401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:36.120475 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 461/909] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:36.127900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3441f0-acd9-48f5-9a47-f140b9a66682 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:36.140040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90b106-08f8-4ce8-9ccb-8591ba45912a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:36.140658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90b106-08f8-4ce8-9ccb-8591ba45912a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:36.140926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90b106-08f8-4ce8-9ccb-8591ba45912a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:36.141198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90b106-08f8-4ce8-9ccb-8591ba45912a None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:36.141801 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90b106-08f8-4ce8-9ccb-8591ba45912a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:36.143408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90b106-08f8-4ce8-9ccb-8591ba45912a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:36.159235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 449/910] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 13:02:35 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:02:36.163632 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:36.245481 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3441f0-acd9-48f5-9a47-f140b9a66682 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:36.287650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d9fe7-6fa6-403a-855e-434ea05a7381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:36.288294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d9fe7-6fa6-403a-855e-434ea05a7381 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:36.288577 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d9fe7-6fa6-403a-855e-434ea05a7381 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:36.288855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d9fe7-6fa6-403a-855e-434ea05a7381 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:36.289654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d9fe7-6fa6-403a-855e-434ea05a7381 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:36.289974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d9fe7-6fa6-403a-855e-434ea05a7381 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:36.291537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d9fe7-6fa6-403a-855e-434ea05a7381 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:36.306222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 462/911] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:36.313063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90b106-08f8-4ce8-9ccb-8591ba45912a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:36.421983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90b106-08f8-4ce8-9ccb-8591ba45912a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:36.429708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:36.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a80f97-6a2d-47ab-aff5-e67cdfacedaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:36.461026 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a80f97-6a2d-47ab-aff5-e67cdfacedaf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:36.461328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a80f97-6a2d-47ab-aff5-e67cdfacedaf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:36.461606 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a80f97-6a2d-47ab-aff5-e67cdfacedaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:36.462336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a80f97-6a2d-47ab-aff5-e67cdfacedaf None admin] RBAC: Adding query filter params (role.id=bd6892add1ee40ac8720e915e153c34f, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=8d343a3db21c47e1a85a739a55559c97) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:36.462647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a80f97-6a2d-47ab-aff5-e67cdfacedaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:36.464133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a80f97-6a2d-47ab-aff5-e67cdfacedaf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:36.471226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 450/912] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:36 2022] GET /identity/v3/role_assignments?user.id=8d343a3db21c47e1a85a739a55559c97&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=bd6892add1ee40ac8720e915e153c34f => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:36.907355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e3497-130d-48f7-89f8-28f2595b667c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:36.907858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e3497-130d-48f7-89f8-28f2595b667c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:36.907858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e3497-130d-48f7-89f8-28f2595b667c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:36.909369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 463/913] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:36.914927 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546baee4-8d80-4d15-b528-9363123a2f26 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:36.915370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546baee4-8d80-4d15-b528-9363123a2f26 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:36.915738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546baee4-8d80-4d15-b528-9363123a2f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:36.935899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-546baee4-8d80-4d15-b528-9363123a2f26 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:36.937441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-546baee4-8d80-4d15-b528-9363123a2f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:37.021452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 451/914] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:37.101769 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382b282-445e-4bbf-9836-7b8d6dad2f82 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:37.102055 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382b282-445e-4bbf-9836-7b8d6dad2f82 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:37.102313 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382b282-445e-4bbf-9836-7b8d6dad2f82 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:37.103935 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 464/915] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:37.109825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7690e-5526-4e75-8f2e-31e1c48714d4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:37.110108 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7690e-5526-4e75-8f2e-31e1c48714d4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:37.110384 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7690e-5526-4e75-8f2e-31e1c48714d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:37.130488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-56d7690e-5526-4e75-8f2e-31e1c48714d4 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:37.131738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-56d7690e-5526-4e75-8f2e-31e1c48714d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:37.207171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 452/916] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:37.302729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382b282-445e-4bbf-9836-7b8d6dad2f82 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:37.399298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:37.412002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382b282-445e-4bbf-9836-7b8d6dad2f82 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:37.447261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfc1db-ccea-48c0-9333-7eb65fb2ff5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:37.447888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfc1db-ccea-48c0-9333-7eb65fb2ff5d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:37.448216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfc1db-ccea-48c0-9333-7eb65fb2ff5d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:37.448487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfc1db-ccea-48c0-9333-7eb65fb2ff5d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:37.449058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bfc1db-ccea-48c0-9333-7eb65fb2ff5d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:37.450470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bfc1db-ccea-48c0-9333-7eb65fb2ff5d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-38bfc1db-ccea-48c0-9333-7eb65fb2ff5d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:37.460399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:37.461369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:37.462205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 465/917] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 13:02:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:37.468054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7690e-5526-4e75-8f2e-31e1c48714d4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:37.500334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:37.507093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfc1db-ccea-48c0-9333-7eb65fb2ff5d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:37.579932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7690e-5526-4e75-8f2e-31e1c48714d4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:37.627951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24874b-0334-444e-b1fa-19a9ecddcb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:37.628862 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24874b-0334-444e-b1fa-19a9ecddcb5a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:37.629171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24874b-0334-444e-b1fa-19a9ecddcb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:37.629448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24874b-0334-444e-b1fa-19a9ecddcb5a None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:37.630233 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24874b-0334-444e-b1fa-19a9ecddcb5a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:37.630670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24874b-0334-444e-b1fa-19a9ecddcb5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:37.632292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24874b-0334-444e-b1fa-19a9ecddcb5a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:37.638931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfc1db-ccea-48c0-9333-7eb65fb2ff5d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:37.639243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 453/918] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 13:02:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:37.652394 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24874b-0334-444e-b1fa-19a9ecddcb5a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:37.680357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3cdf3-7065-4f12-ab90-a9d0056a5e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:37.681757 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3cdf3-7065-4f12-ab90-a9d0056a5e94 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:37.682494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3cdf3-7065-4f12-ab90-a9d0056a5e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:37.682932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3cdf3-7065-4f12-ab90-a9d0056a5e94 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:37.683687 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3cdf3-7065-4f12-ab90-a9d0056a5e94 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:37.686120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3cdf3-7065-4f12-ab90-a9d0056a5e94 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7bf3cdf3-7065-4f12-ab90-a9d0056a5e94 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:37.697314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:37.698335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 13:02:37.699275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:37.702058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 466/919] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:02:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:37.715132 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3cdf3-7065-4f12-ab90-a9d0056a5e94 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:37.734963 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:37.781073 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:37.791859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24874b-0334-444e-b1fa-19a9ecddcb5a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:37.832619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3cdf3-7065-4f12-ab90-a9d0056a5e94 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:37.846068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a8ce8-4524-49a6-b4fd-ba15f1c44fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:37.847321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a8ce8-4524-49a6-b4fd-ba15f1c44fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:37.847973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a8ce8-4524-49a6-b4fd-ba15f1c44fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:37.848338 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a8ce8-4524-49a6-b4fd-ba15f1c44fb7 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:37.849095 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a8ce8-4524-49a6-b4fd-ba15f1c44fb7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:37.849492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a8ce8-4524-49a6-b4fd-ba15f1c44fb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:37.851778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a8ce8-4524-49a6-b4fd-ba15f1c44fb7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:37.863199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 454/920] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 13:02:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:37.865093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a8ce8-4524-49a6-b4fd-ba15f1c44fb7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:37.880771 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0633e-10bd-4406-b5dd-62def5f9b570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:37.881383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0633e-10bd-4406-b5dd-62def5f9b570 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:37.881666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0633e-10bd-4406-b5dd-62def5f9b570 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:37.881938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0633e-10bd-4406-b5dd-62def5f9b570 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:37.882570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0633e-10bd-4406-b5dd-62def5f9b570 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:37.882875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0633e-10bd-4406-b5dd-62def5f9b570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:37.885029 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0633e-10bd-4406-b5dd-62def5f9b570 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:37.890317 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 467/921] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:02:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:37.895678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0633e-10bd-4406-b5dd-62def5f9b570 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:37.984302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:37.988071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a8ce8-4524-49a6-b4fd-ba15f1c44fb7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:38.004712 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0633e-10bd-4406-b5dd-62def5f9b570 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:38.025393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61a95b-5922-41ee-bb50-5ddcf5a9b928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:38.026256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61a95b-5922-41ee-bb50-5ddcf5a9b928 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:38.026686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61a95b-5922-41ee-bb50-5ddcf5a9b928 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:38.027121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61a95b-5922-41ee-bb50-5ddcf5a9b928 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:38.027959 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61a95b-5922-41ee-bb50-5ddcf5a9b928 None admin] RBAC: Adding query filter params (interface=public, service_id=e44a78cfc2694ab983a953cdd82e55c2, region_id=RegionOne) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:38.028407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61a95b-5922-41ee-bb50-5ddcf5a9b928 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:38.030988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61a95b-5922-41ee-bb50-5ddcf5a9b928 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:38.041576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 455/922] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 13:02:37 2022] GET /identity/v3/endpoints?service_id=e44a78cfc2694ab983a953cdd82e55c2&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:38.042362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61a95b-5922-41ee-bb50-5ddcf5a9b928 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:38.046367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e9713-8da3-4393-93df-c5db211aa62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:38.046986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e9713-8da3-4393-93df-c5db211aa62a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:38.047340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e9713-8da3-4393-93df-c5db211aa62a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:38.047584 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e9713-8da3-4393-93df-c5db211aa62a None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:38.048227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e9713-8da3-4393-93df-c5db211aa62a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:38.048526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e9713-8da3-4393-93df-c5db211aa62a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:38.049892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e9713-8da3-4393-93df-c5db211aa62a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:38.060604 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 468/923] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 9 13:02:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:38.061853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e9713-8da3-4393-93df-c5db211aa62a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:38.072696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:38.160243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61a95b-5922-41ee-bb50-5ddcf5a9b928 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:38.168678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e9713-8da3-4393-93df-c5db211aa62a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:38.199002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a78999-b21b-46da-9e09-cb5b15944b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:38.200012 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a78999-b21b-46da-9e09-cb5b15944b4f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:38.200546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a78999-b21b-46da-9e09-cb5b15944b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:38.201685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a78999-b21b-46da-9e09-cb5b15944b4f None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:38.202633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a78999-b21b-46da-9e09-cb5b15944b4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:38.204476 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a78999-b21b-46da-9e09-cb5b15944b4f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:38.205932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493a81c-1e58-4839-ae0e-8ebfb4f28848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:38.206545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493a81c-1e58-4839-ae0e-8ebfb4f28848 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:38.206826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493a81c-1e58-4839-ae0e-8ebfb4f28848 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:38.207142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493a81c-1e58-4839-ae0e-8ebfb4f28848 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:38.207829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493a81c-1e58-4839-ae0e-8ebfb4f28848 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:38.209184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493a81c-1e58-4839-ae0e-8ebfb4f28848 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:38.217459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 456/924] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 13:02:38 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:02:38.245890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 469/925] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:38.494030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cb78d-a259-410c-9036-ac1f95b818f7 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:38.494539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cb78d-a259-410c-9036-ac1f95b818f7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:38.494643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cb78d-a259-410c-9036-ac1f95b818f7 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:38.496261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 457/926] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:38.502592 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cdcd8-de10-4895-a409-8c62f6fd9d2f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:38.502878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cdcd8-de10-4895-a409-8c62f6fd9d2f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:38.503226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cdcd8-de10-4895-a409-8c62f6fd9d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:38.509943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:38.526167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-380cdcd8-de10-4895-a409-8c62f6fd9d2f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:38.527421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-380cdcd8-de10-4895-a409-8c62f6fd9d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:38.613018 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 470/927] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:38.726488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bef5ee-c76c-434f-bfe6-978555701fe2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:38.726843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bef5ee-c76c-434f-bfe6-978555701fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:38.727032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bef5ee-c76c-434f-bfe6-978555701fe2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:38.728670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 458/928] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:38.733937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e15ef0-63c7-494c-899c-b8521f239062 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:38.734225 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e15ef0-63c7-494c-899c-b8521f239062 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:38.734502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e15ef0-63c7-494c-899c-b8521f239062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:38.754190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-47e15ef0-63c7-494c-899c-b8521f239062 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:38.755333 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-47e15ef0-63c7-494c-899c-b8521f239062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:38.829921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 471/929] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:38.888671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bef5ee-c76c-434f-bfe6-978555701fe2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:38.982789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:39.001303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bef5ee-c76c-434f-bfe6-978555701fe2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:39.039331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098aa63-bab2-43ce-a8fe-21aecc1e6ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:39.039931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098aa63-bab2-43ce-a8fe-21aecc1e6ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:39.040205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098aa63-bab2-43ce-a8fe-21aecc1e6ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:39.040505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098aa63-bab2-43ce-a8fe-21aecc1e6ab4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:39.041095 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098aa63-bab2-43ce-a8fe-21aecc1e6ab4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:39.042460 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098aa63-bab2-43ce-a8fe-21aecc1e6ab4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2098aa63-bab2-43ce-a8fe-21aecc1e6ab4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:39.052759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:39.053735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:39.054651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 459/930] 10.222.0.17 () {56 vars in 1132 bytes} [Fri Sep 9 13:02:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:39.062536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e15ef0-63c7-494c-899c-b8521f239062 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:39.105063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098aa63-bab2-43ce-a8fe-21aecc1e6ab4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:39.139393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:39.194813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e15ef0-63c7-494c-899c-b8521f239062 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:39.238661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098aa63-bab2-43ce-a8fe-21aecc1e6ab4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:39.239630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34803779-7d61-4718-9b68-faacdb90523f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:39.240254 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34803779-7d61-4718-9b68-faacdb90523f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:39.240536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34803779-7d61-4718-9b68-faacdb90523f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:39.240807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34803779-7d61-4718-9b68-faacdb90523f None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:39.241562 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34803779-7d61-4718-9b68-faacdb90523f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:39.241870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34803779-7d61-4718-9b68-faacdb90523f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:39.243389 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34803779-7d61-4718-9b68-faacdb90523f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:39.253333 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 472/931] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 13:02:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:39.254248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34803779-7d61-4718-9b68-faacdb90523f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:39.297741 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46916178-eeed-43e6-a6e3-7b60c6a04acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:39.299486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46916178-eeed-43e6-a6e3-7b60c6a04acf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:39.300658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46916178-eeed-43e6-a6e3-7b60c6a04acf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:39.301379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46916178-eeed-43e6-a6e3-7b60c6a04acf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:39.321343 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46916178-eeed-43e6-a6e3-7b60c6a04acf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-46916178-eeed-43e6-a6e3-7b60c6a04acf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:39.336744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:39.337845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:39.338703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:39.338703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:39.338703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:39.338703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:39.338703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:39.338703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:39.338703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:39.338703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:39.338703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:39.338703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:39.338703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 13:02:39.338703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:39.343082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 460/932] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 9 13:02:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:39.346359 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46916178-eeed-43e6-a6e3-7b60c6a04acf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:39.358776 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:39.412692 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34803779-7d61-4718-9b68-faacdb90523f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:39.443542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:39.457261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828373b-2089-4a15-a7bf-5405e73b9131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:39.458013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828373b-2089-4a15-a7bf-5405e73b9131 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:39.458386 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828373b-2089-4a15-a7bf-5405e73b9131 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:39.458778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828373b-2089-4a15-a7bf-5405e73b9131 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:39.459553 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828373b-2089-4a15-a7bf-5405e73b9131 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:39.460032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828373b-2089-4a15-a7bf-5405e73b9131 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:39.461411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828373b-2089-4a15-a7bf-5405e73b9131 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:39.471633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 473/933] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 13:02:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:39.472686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828373b-2089-4a15-a7bf-5405e73b9131 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:39.481787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46916178-eeed-43e6-a6e3-7b60c6a04acf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:39.518928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d620519-90e6-40a7-9066-7f0a270012be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:39.520340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d620519-90e6-40a7-9066-7f0a270012be None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:39.520758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d620519-90e6-40a7-9066-7f0a270012be None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:39.521258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d620519-90e6-40a7-9066-7f0a270012be None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:39.522115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d620519-90e6-40a7-9066-7f0a270012be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:39.522608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d620519-90e6-40a7-9066-7f0a270012be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:39.524290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d620519-90e6-40a7-9066-7f0a270012be None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:39.534232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 461/934] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 13:02:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:39.540754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d620519-90e6-40a7-9066-7f0a270012be None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:39.598488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828373b-2089-4a15-a7bf-5405e73b9131 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:39.637023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:39.642764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d070d1-2493-427e-9dcb-e8aeadac6d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:39.643488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d070d1-2493-427e-9dcb-e8aeadac6d61 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:39.643835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d070d1-2493-427e-9dcb-e8aeadac6d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:39.644176 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d070d1-2493-427e-9dcb-e8aeadac6d61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:39.644920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d070d1-2493-427e-9dcb-e8aeadac6d61 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:39.646362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d070d1-2493-427e-9dcb-e8aeadac6d61 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:39.670268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 474/935] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:39.679863 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d620519-90e6-40a7-9066-7f0a270012be None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:39.729871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536645a7-e0ff-45ec-a6b3-a78fc49fbde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:39.730996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536645a7-e0ff-45ec-a6b3-a78fc49fbde5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:39.732065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536645a7-e0ff-45ec-a6b3-a78fc49fbde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:39.732560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536645a7-e0ff-45ec-a6b3-a78fc49fbde5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:39.746295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536645a7-e0ff-45ec-a6b3-a78fc49fbde5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:39.748578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536645a7-e0ff-45ec-a6b3-a78fc49fbde5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-536645a7-e0ff-45ec-a6b3-a78fc49fbde5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:39.766658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:39.767799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:39.768756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:39.768756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:39.768756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:39.771146 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 462/936] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:39.781698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d070d1-2493-427e-9dcb-e8aeadac6d61 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:39.834662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:39.896769 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d070d1-2493-427e-9dcb-e8aeadac6d61 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:39.933004 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8673dd-f77e-4f54-b0eb-000d8e3ad535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:39.933862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8673dd-f77e-4f54-b0eb-000d8e3ad535 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:39.934630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8673dd-f77e-4f54-b0eb-000d8e3ad535 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:39.935125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8673dd-f77e-4f54-b0eb-000d8e3ad535 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:39.936056 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8673dd-f77e-4f54-b0eb-000d8e3ad535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:39.936508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8673dd-f77e-4f54-b0eb-000d8e3ad535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:39.938022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8673dd-f77e-4f54-b0eb-000d8e3ad535 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:39.951607 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 475/937] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:39.957453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536645a7-e0ff-45ec-a6b3-a78fc49fbde5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:40.067239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536645a7-e0ff-45ec-a6b3-a78fc49fbde5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:40.088818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:40.105199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a18ca-fda2-4aff-a756-4e1ab7f75a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:40.105842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a18ca-fda2-4aff-a756-4e1ab7f75a59 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:40.106189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a18ca-fda2-4aff-a756-4e1ab7f75a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:40.106490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a18ca-fda2-4aff-a756-4e1ab7f75a59 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:40.114577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a18ca-fda2-4aff-a756-4e1ab7f75a59 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:40.116305 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a18ca-fda2-4aff-a756-4e1ab7f75a59 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-651a18ca-fda2-4aff-a756-4e1ab7f75a59 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:40.128848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:40.129827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:40.130802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:40.130802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:40.130802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:40.130802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:40.130802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:40.130802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:40.130802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:40.130802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:40.130802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 13:02:40.130802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:40.130802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 463/938] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 13:02:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:40.137963 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8673dd-f77e-4f54-b0eb-000d8e3ad535 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:40.245875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8673dd-f77e-4f54-b0eb-000d8e3ad535 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:40.248437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:40.282924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d7271-f833-4ad8-ac14-49059ce36a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:40.283569 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d7271-f833-4ad8-ac14-49059ce36a52 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:40.283848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d7271-f833-4ad8-ac14-49059ce36a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:40.284120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d7271-f833-4ad8-ac14-49059ce36a52 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:40.284849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d7271-f833-4ad8-ac14-49059ce36a52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:40.285150 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d7271-f833-4ad8-ac14-49059ce36a52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:40.286571 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d7271-f833-4ad8-ac14-49059ce36a52 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:40.304726 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 476/939] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 9 13:02:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:40.311346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a18ca-fda2-4aff-a756-4e1ab7f75a59 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:40.425904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a18ca-fda2-4aff-a756-4e1ab7f75a59 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:40.462885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745305a-caba-4a51-b43b-b16e583de456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:40.463554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745305a-caba-4a51-b43b-b16e583de456 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:40.463832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745305a-caba-4a51-b43b-b16e583de456 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:40.464109 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745305a-caba-4a51-b43b-b16e583de456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:40.475848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745305a-caba-4a51-b43b-b16e583de456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:40.477346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745305a-caba-4a51-b43b-b16e583de456 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1745305a-caba-4a51-b43b-b16e583de456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:40.495034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:40.496104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:40.497199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:40.497199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:40.497199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:40.497199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 464/940] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:40.502609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d7271-f833-4ad8-ac14-49059ce36a52 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:40.519647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa7421-03d5-4dd7-892b-c59db2f0d8d6 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:40.520159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa7421-03d5-4dd7-892b-c59db2f0d8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:40.520159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa7421-03d5-4dd7-892b-c59db2f0d8d6 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:40.521735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 465/941] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:40.526675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d40918-e637-411f-812f-b1c4f82e303c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:40.526937 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d40918-e637-411f-812f-b1c4f82e303c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:40.527250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d40918-e637-411f-812f-b1c4f82e303c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:40.546760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-98d40918-e637-411f-812f-b1c4f82e303c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:40.548004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-98d40918-e637-411f-812f-b1c4f82e303c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:40.582852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:40.628870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d7271-f833-4ad8-ac14-49059ce36a52 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:40.635951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 466/942] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:40.637454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33118fcb-6357-4855-affd-f75aefc41111 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:40.638299 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33118fcb-6357-4855-affd-f75aefc41111 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:40.638722 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33118fcb-6357-4855-affd-f75aefc41111 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:40.640351 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 467/943] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:40.645529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66064b-6690-45dd-954a-2d7632c403dc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:40.645807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66064b-6690-45dd-954a-2d7632c403dc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:40.646077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66064b-6690-45dd-954a-2d7632c403dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:40.664294 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9a66064b-6690-45dd-954a-2d7632c403dc None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:40.665529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9a66064b-6690-45dd-954a-2d7632c403dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:40.675065 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495dfa6-84e2-44a8-beb8-242a6be500d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:40.676111 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495dfa6-84e2-44a8-beb8-242a6be500d9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:40.676604 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495dfa6-84e2-44a8-beb8-242a6be500d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:40.677548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495dfa6-84e2-44a8-beb8-242a6be500d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:40.678817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495dfa6-84e2-44a8-beb8-242a6be500d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:40.679387 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495dfa6-84e2-44a8-beb8-242a6be500d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:40.681493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495dfa6-84e2-44a8-beb8-242a6be500d9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:40.698940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 477/944] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:40.708083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495dfa6-84e2-44a8-beb8-242a6be500d9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:40.731620 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:40.759482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 468/945] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:40.811695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495dfa6-84e2-44a8-beb8-242a6be500d9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:40.845864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58759aa-e8ae-48fd-add6-8e1f091f40a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:40.846494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58759aa-e8ae-48fd-add6-8e1f091f40a4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:40.846770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58759aa-e8ae-48fd-add6-8e1f091f40a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:40.847039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58759aa-e8ae-48fd-add6-8e1f091f40a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:40.853701 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58759aa-e8ae-48fd-add6-8e1f091f40a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:40.855193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58759aa-e8ae-48fd-add6-8e1f091f40a4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e58759aa-e8ae-48fd-add6-8e1f091f40a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:40.866544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:40.867477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:40.868384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:40.868384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:40.868384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:40.868384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:40.868384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:40.868384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:40.868384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:40.868384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:40.868384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 478/946] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:40.874447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66064b-6690-45dd-954a-2d7632c403dc None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:40.907758 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58759aa-e8ae-48fd-add6-8e1f091f40a4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:40.974551 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:40.984768 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66064b-6690-45dd-954a-2d7632c403dc None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:41.006432 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:41.025968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31cdbd4-a508-47fb-a8f3-2a8b109e9224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:41.026515 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cdbd4-a508-47fb-a8f3-2a8b109e9224 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:41.026794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cdbd4-a508-47fb-a8f3-2a8b109e9224 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:41.027082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cdbd4-a508-47fb-a8f3-2a8b109e9224 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:41.027778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31cdbd4-a508-47fb-a8f3-2a8b109e9224 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:41.028111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31cdbd4-a508-47fb-a8f3-2a8b109e9224 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:41.030042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31cdbd4-a508-47fb-a8f3-2a8b109e9224 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:41.031203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58759aa-e8ae-48fd-add6-8e1f091f40a4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:41.047740 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 469/947] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:41.048748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31cdbd4-a508-47fb-a8f3-2a8b109e9224 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:41.068021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111758d-5247-45be-912d-2f9bc89ea5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:41.068654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111758d-5247-45be-912d-2f9bc89ea5bb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:41.068929 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111758d-5247-45be-912d-2f9bc89ea5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:41.069195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111758d-5247-45be-912d-2f9bc89ea5bb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:41.069771 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111758d-5247-45be-912d-2f9bc89ea5bb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:41.071155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111758d-5247-45be-912d-2f9bc89ea5bb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-c111758d-5247-45be-912d-2f9bc89ea5bb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:41.080810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:41.081819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:41.082671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 479/948] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 13:02:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:41.083528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111758d-5247-45be-912d-2f9bc89ea5bb None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:41.164682 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31cdbd4-a508-47fb-a8f3-2a8b109e9224 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:41.193916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111758d-5247-45be-912d-2f9bc89ea5bb None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:41.205546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d83b6-3ce9-4932-8247-549b2115e92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:41.206431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d83b6-3ce9-4932-8247-549b2115e92d None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:41.206709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d83b6-3ce9-4932-8247-549b2115e92d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:41.206986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d83b6-3ce9-4932-8247-549b2115e92d None admin] PATH_INFO: `/v3/projects/2b842ac074b44e148c15882a7964aa13/users/8d343a3db21c47e1a85a739a55559c97/roles/bd6892add1ee40ac8720e915e153c34f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:41.209243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d83b6-3ce9-4932-8247-549b2115e92d None admin] RBAC: Authorizing `identity:create_grant(project_id=2b842ac074b44e148c15882a7964aa13, user_id=8d343a3db21c47e1a85a739a55559c97, role_id=bd6892add1ee40ac8720e915e153c34f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:41.210609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d83b6-3ce9-4932-8247-549b2115e92d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:41.224749 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 470/949] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:02:41 2022] PUT /identity/v3/projects/2b842ac074b44e148c15882a7964aa13/users/8d343a3db21c47e1a85a739a55559c97/roles/bd6892add1ee40ac8720e915e153c34f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:41.225644 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d83b6-3ce9-4932-8247-549b2115e92d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:41.245994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:41.247761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57cf437-61db-4b9b-b15a-6c7e25142d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:41.248540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cf437-61db-4b9b-b15a-6c7e25142d09 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:41.248748 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cf437-61db-4b9b-b15a-6c7e25142d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:41.249027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cf437-61db-4b9b-b15a-6c7e25142d09 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:41.249744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57cf437-61db-4b9b-b15a-6c7e25142d09 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:41.251989 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57cf437-61db-4b9b-b15a-6c7e25142d09 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:41.255857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: WARNING keystone.server.flask.common [None req-b57cf437-61db-4b9b-b15a-6c7e25142d09 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 13:02:41.274514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:41.277729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.sql.core [None req-b57cf437-61db-4b9b-b15a-6c7e25142d09 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 13:02:41.277729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 13:02:41.277729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [parameters: {'id': '3fd58acd29d341f89e5aaee2496fad8a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 13:02:41.277729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81603) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 13:02:41.299859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-b57cf437-61db-4b9b-b15a-6c7e25142d09 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3fd58acd29d341f89e5aaee2496fad8a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3fd58acd29d341f89e5aaee2496fad8a. Sep 09 13:02:41.299859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:41.299859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 13:02:41.299859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 13:02:41.299859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 13:02:41.299859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 13:02:41.299859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 13:02:41.299859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 13:02:41.299859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 13:02:41.299859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application conn.query(q) Sep 09 13:02:41.299859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 13:02:41.299859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 13:02:41.299859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result.read() Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 13:02:41.300809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application next(self.gen) Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application yield resource Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application next(self.gen) Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise self.value Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application session.commit() Sep 09 13:02:41.301780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.session.flush() Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application compat.raise_( Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 13:02:41.302681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 13:02:41.303626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application conn.query(q) Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result.read() Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 13:02:41.304447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [parameters: {'id': '3fd58acd29d341f89e5aaee2496fad8a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:41.305319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:41.307643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:41.310186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:41.310186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 13:02:41.310186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 13:02:41.310186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:41.310186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:41.310186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 13:02:41.310186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 13:02:41.310186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3fd58acd29d341f89e5aaee2496fad8a. Sep 09 13:02:41.310186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:41.310186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 480/950] 10.222.0.17 () {60 vars in 1140 bytes} [Fri Sep 9 13:02:41 2022] POST /identity/v3/projects => generated 262 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:41.310186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57cf437-61db-4b9b-b15a-6c7e25142d09 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:41.401265 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d83b6-3ce9-4932-8247-549b2115e92d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:41.432873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57cf437-61db-4b9b-b15a-6c7e25142d09 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:41.441315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543643a-1d83-4224-92f1-3a99b50bbe7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:41.441929 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543643a-1d83-4224-92f1-3a99b50bbe7d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:41.442203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543643a-1d83-4224-92f1-3a99b50bbe7d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:41.442790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543643a-1d83-4224-92f1-3a99b50bbe7d None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:41.443123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543643a-1d83-4224-92f1-3a99b50bbe7d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:41.443430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543643a-1d83-4224-92f1-3a99b50bbe7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:41.444813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543643a-1d83-4224-92f1-3a99b50bbe7d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:41.450731 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 471/951] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 13:02:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:41.456338 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543643a-1d83-4224-92f1-3a99b50bbe7d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:41.475327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63003473-3265-498b-99ae-53810613b280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:41.476513 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63003473-3265-498b-99ae-53810613b280 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:41.476938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63003473-3265-498b-99ae-53810613b280 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:41.477394 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63003473-3265-498b-99ae-53810613b280 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:41.485709 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63003473-3265-498b-99ae-53810613b280 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:41.488297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63003473-3265-498b-99ae-53810613b280 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-63003473-3265-498b-99ae-53810613b280 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:41.503195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:41.504472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:41.505643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:41.505643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:41.505643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:41.505643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:41.505643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:41.505643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:41.505643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:41.505643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:41.507646 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 481/952] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:41.515078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63003473-3265-498b-99ae-53810613b280 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:41.545572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:41.570482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543643a-1d83-4224-92f1-3a99b50bbe7d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:41.590563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:41.612742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff2531-7c2f-4b2a-a896-fea1e519e45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:41.613611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff2531-7c2f-4b2a-a896-fea1e519e45f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:41.614121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff2531-7c2f-4b2a-a896-fea1e519e45f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:41.614589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff2531-7c2f-4b2a-a896-fea1e519e45f None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:41.615431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff2531-7c2f-4b2a-a896-fea1e519e45f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:41.615919 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff2531-7c2f-4b2a-a896-fea1e519e45f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:41.617754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff2531-7c2f-4b2a-a896-fea1e519e45f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:41.624448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 472/953] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 13:02:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:41.630141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff2531-7c2f-4b2a-a896-fea1e519e45f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:41.645782 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63003473-3265-498b-99ae-53810613b280 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:41.691777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f31cff-c62c-4af3-9c28-ab0621f92737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:41.693359 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f31cff-c62c-4af3-9c28-ab0621f92737 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:41.694229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f31cff-c62c-4af3-9c28-ab0621f92737 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:41.694794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f31cff-c62c-4af3-9c28-ab0621f92737 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:41.695640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f31cff-c62c-4af3-9c28-ab0621f92737 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:41.696146 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f31cff-c62c-4af3-9c28-ab0621f92737 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:41.697720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f31cff-c62c-4af3-9c28-ab0621f92737 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:41.731608 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 482/954] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 13:02:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:41.776561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff2531-7c2f-4b2a-a896-fea1e519e45f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:41.816750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9bebf-9359-4554-8292-fd4a26f5a11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:41.817284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9bebf-9359-4554-8292-fd4a26f5a11e None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:41.817561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9bebf-9359-4554-8292-fd4a26f5a11e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:41.817829 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9bebf-9359-4554-8292-fd4a26f5a11e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:41.818511 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9bebf-9359-4554-8292-fd4a26f5a11e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:41.819939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9bebf-9359-4554-8292-fd4a26f5a11e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:41.831137 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:41.851953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 473/955] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:02:42.066946 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425ceda-0c55-4cac-936f-14540aad56ac None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:42.067355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425ceda-0c55-4cac-936f-14540aad56ac None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:42.067459 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425ceda-0c55-4cac-936f-14540aad56ac None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:42.069121 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 483/956] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:42.074611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e3bfd-20cb-4ffb-a4af-66c9cd3166c4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:42.074899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e3bfd-20cb-4ffb-a4af-66c9cd3166c4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:42.075738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e3bfd-20cb-4ffb-a4af-66c9cd3166c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:42.094661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-041e3bfd-20cb-4ffb-a4af-66c9cd3166c4 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:42.095918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-041e3bfd-20cb-4ffb-a4af-66c9cd3166c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:42.178793 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 474/957] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:42.444022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425ceda-0c55-4cac-936f-14540aad56ac None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:42.549749 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425ceda-0c55-4cac-936f-14540aad56ac None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:42.571493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:42.586582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d25b40-de04-4f6f-921a-887ed47fd42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:42.587251 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d25b40-de04-4f6f-921a-887ed47fd42b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:42.587533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d25b40-de04-4f6f-921a-887ed47fd42b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:42.587810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d25b40-de04-4f6f-921a-887ed47fd42b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:42.588400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d25b40-de04-4f6f-921a-887ed47fd42b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:42.589766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d25b40-de04-4f6f-921a-887ed47fd42b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-87d25b40-de04-4f6f-921a-887ed47fd42b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:42.603075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:42.604224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 13:02:42.605094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:42.605931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 484/958] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:42.607171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e3bfd-20cb-4ffb-a4af-66c9cd3166c4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:42.719947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e3bfd-20cb-4ffb-a4af-66c9cd3166c4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:42.744840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:42.756822 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05766758-fd7d-49f7-a284-146dae08cf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:42.757440 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05766758-fd7d-49f7-a284-146dae08cf7d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:42.757716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05766758-fd7d-49f7-a284-146dae08cf7d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:42.757989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05766758-fd7d-49f7-a284-146dae08cf7d None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:42.758619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05766758-fd7d-49f7-a284-146dae08cf7d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:42.758924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05766758-fd7d-49f7-a284-146dae08cf7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:42.760340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05766758-fd7d-49f7-a284-146dae08cf7d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:42.766371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 475/959] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 13:02:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:42.771761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d25b40-de04-4f6f-921a-887ed47fd42b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:42.877601 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d25b40-de04-4f6f-921a-887ed47fd42b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:42.911890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42180018-8f33-476b-9553-afe10c8415e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:42.912502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42180018-8f33-476b-9553-afe10c8415e3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:42.912780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42180018-8f33-476b-9553-afe10c8415e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:42.913054 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42180018-8f33-476b-9553-afe10c8415e3 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:42.913684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42180018-8f33-476b-9553-afe10c8415e3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:42.913989 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42180018-8f33-476b-9553-afe10c8415e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:42.915376 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42180018-8f33-476b-9553-afe10c8415e3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:42.920869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 485/960] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 13:02:42 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:43.575513 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ef8c1-03d8-49d6-a4b5-28aad1b73969 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:43.576436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ef8c1-03d8-49d6-a4b5-28aad1b73969 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:43.576943 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ef8c1-03d8-49d6-a4b5-28aad1b73969 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:43.578753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 476/961] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:43.584433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b163ad-03c8-44a2-85ca-58c2fac7c127 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:43.585298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b163ad-03c8-44a2-85ca-58c2fac7c127 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:43.585771 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b163ad-03c8-44a2-85ca-58c2fac7c127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:43.613603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-92b163ad-03c8-44a2-85ca-58c2fac7c127 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:43.615282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-92b163ad-03c8-44a2-85ca-58c2fac7c127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:43.663278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:43.688483 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 486/962] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:43.967966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ef8c1-03d8-49d6-a4b5-28aad1b73969 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:44.051417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc54c8-a0f4-4ca3-ad1f-21d8aab54dd1 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:44.051859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc54c8-a0f4-4ca3-ad1f-21d8aab54dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:44.052225 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc54c8-a0f4-4ca3-ad1f-21d8aab54dd1 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:44.054110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 487/963] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:02:44.060944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb57689-fe7e-4071-a1d1-9746341b8f56 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:44.061605 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb57689-fe7e-4071-a1d1-9746341b8f56 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:44.062087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb57689-fe7e-4071-a1d1-9746341b8f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:44.082443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dbb57689-fe7e-4071-a1d1-9746341b8f56 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:44.082665 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ef8c1-03d8-49d6-a4b5-28aad1b73969 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:44.084308 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dbb57689-fe7e-4071-a1d1-9746341b8f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:44.121055 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52de30c-3743-4fbb-94ed-ce2d09631b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:44.122656 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52de30c-3743-4fbb-94ed-ce2d09631b2b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:44.123174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52de30c-3743-4fbb-94ed-ce2d09631b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:44.123449 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52de30c-3743-4fbb-94ed-ce2d09631b2b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:44.134691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52de30c-3743-4fbb-94ed-ce2d09631b2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c52de30c-3743-4fbb-94ed-ce2d09631b2b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:44.140505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:44.141475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:44.142330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:44.142330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:44.142330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:44.142330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:44.142330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:44.142330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:44.142330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:44.142330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:44.142330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:44.142330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:44.142330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 13:02:44.142330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:44.142330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 477/964] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 9 13:02:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:44.147595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52de30c-3743-4fbb-94ed-ce2d09631b2b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:44.173996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 488/965] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:44.190737 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28182e17-f3b1-4153-9d40-28bbf4376c34 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:44.191013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28182e17-f3b1-4153-9d40-28bbf4376c34 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:44.191326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28182e17-f3b1-4153-9d40-28bbf4376c34 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:44.192924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 489/966] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:02:44.197797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f7b76-8982-4224-bdcb-2ef42df66bbf None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:44.198084 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f7b76-8982-4224-bdcb-2ef42df66bbf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:44.198361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f7b76-8982-4224-bdcb-2ef42df66bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:44.209035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:44.220842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f45f7b76-8982-4224-bdcb-2ef42df66bbf None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:44.221958 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f45f7b76-8982-4224-bdcb-2ef42df66bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:44.279732 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52de30c-3743-4fbb-94ed-ce2d09631b2b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:44.299239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 490/967] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:44.317857 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe74602-a3d8-43dd-84ab-828f1e3e825b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:44.318743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe74602-a3d8-43dd-84ab-828f1e3e825b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:44.319283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe74602-a3d8-43dd-84ab-828f1e3e825b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:44.319734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe74602-a3d8-43dd-84ab-828f1e3e825b None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:44.320631 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe74602-a3d8-43dd-84ab-828f1e3e825b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:44.321106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe74602-a3d8-43dd-84ab-828f1e3e825b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:44.322676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe74602-a3d8-43dd-84ab-828f1e3e825b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:44.332605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 478/968] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 13:02:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:44.338487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f7b76-8982-4224-bdcb-2ef42df66bbf None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:44.383849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:44.445788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f7b76-8982-4224-bdcb-2ef42df66bbf None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:44.462415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe74602-a3d8-43dd-84ab-828f1e3e825b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:44.480811 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343a3f2-36d3-44cc-9425-00c7e222daa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:44.482009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a3f2-36d3-44cc-9425-00c7e222daa8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:44.482297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a3f2-36d3-44cc-9425-00c7e222daa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:44.482546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a3f2-36d3-44cc-9425-00c7e222daa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:44.493291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343a3f2-36d3-44cc-9425-00c7e222daa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:44.494794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343a3f2-36d3-44cc-9425-00c7e222daa8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-1343a3f2-36d3-44cc-9425-00c7e222daa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:44.510776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:44.511812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:44.513639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:44.513639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:44.513639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:44.513639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 491/969] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:44.519493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343a3f2-36d3-44cc-9425-00c7e222daa8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:44.596920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe74602-a3d8-43dd-84ab-828f1e3e825b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:44.609647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:44.633779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:44.639563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cc490-d273-4624-a772-c31b8f4799b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:44.640185 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cc490-d273-4624-a772-c31b8f4799b7 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:44.640467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cc490-d273-4624-a772-c31b8f4799b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:44.640774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cc490-d273-4624-a772-c31b8f4799b7 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:44.641436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955cc490-d273-4624-a772-c31b8f4799b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:44.642881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955cc490-d273-4624-a772-c31b8f4799b7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:44.644311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343a3f2-36d3-44cc-9425-00c7e222daa8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:44.645475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: WARNING keystone.server.flask.common [None req-955cc490-d273-4624-a772-c31b8f4799b7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 13:02:44.658379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.sql.core [None req-955cc490-d273-4624-a772-c31b8f4799b7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 13:02:44.658379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 13:02:44.658379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [parameters: {'user_id': '75ece184cf8c46bfa981a17a5ed984ec', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 13:02:44.658379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81602) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-955cc490-d273-4624-a772-c31b8f4799b7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application conn.query(q) Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result.read() Sep 09 13:02:44.665603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 13:02:44.666711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application next(self.gen) Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application yield resource Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application next(self.gen) Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise self.value Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application session.commit() Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 13:02:44.667869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.session.flush() Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application compat.raise_( Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 13:02:44.669542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 13:02:44.670565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application conn.query(q) Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result.read() Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 13:02:44.671488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [parameters: {'user_id': '75ece184cf8c46bfa981a17a5ed984ec', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:44.672436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 13:02:44.673313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 13:02:44.674519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 13:02:44.674519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 13:02:44.674519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 13:02:44.674519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 13:02:44.674519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 13:02:44.674519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:44.674519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 479/970] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 9 13:02:44 2022] POST /identity/v3/users => generated 160 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:44.674519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cc490-d273-4624-a772-c31b8f4799b7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:44.690194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b2b67-738d-4034-9cbc-5ff19de18be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:44.690813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b2b67-738d-4034-9cbc-5ff19de18be0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:44.691195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b2b67-738d-4034-9cbc-5ff19de18be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:44.691475 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b2b67-738d-4034-9cbc-5ff19de18be0 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:44.692112 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b2b67-738d-4034-9cbc-5ff19de18be0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:44.692656 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b2b67-738d-4034-9cbc-5ff19de18be0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:44.693776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b2b67-738d-4034-9cbc-5ff19de18be0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:44.707102 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 492/971] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:44.707102 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b2b67-738d-4034-9cbc-5ff19de18be0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:44.781579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cc490-d273-4624-a772-c31b8f4799b7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:44.816437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e89f6-ca88-44bc-9574-c0e2a4899324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:44.817064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e89f6-ca88-44bc-9574-c0e2a4899324 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:44.817340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e89f6-ca88-44bc-9574-c0e2a4899324 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:44.817607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e89f6-ca88-44bc-9574-c0e2a4899324 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:44.818175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e89f6-ca88-44bc-9574-c0e2a4899324 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:44.819188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b2b67-738d-4034-9cbc-5ff19de18be0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:44.819771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e89f6-ca88-44bc-9574-c0e2a4899324 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-aa7e89f6-ca88-44bc-9574-c0e2a4899324 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:44.829197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:44.830256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:44.831176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 480/972] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 13:02:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:44.831986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e89f6-ca88-44bc-9574-c0e2a4899324 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:44.860361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca2ae2-8ea6-44e2-9cdb-4ea4738cedc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:44.861159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca2ae2-8ea6-44e2-9cdb-4ea4738cedc2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:44.861767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca2ae2-8ea6-44e2-9cdb-4ea4738cedc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:44.862043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca2ae2-8ea6-44e2-9cdb-4ea4738cedc2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:44.869252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca2ae2-8ea6-44e2-9cdb-4ea4738cedc2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:44.870673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:44.870791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca2ae2-8ea6-44e2-9cdb-4ea4738cedc2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-66ca2ae2-8ea6-44e2-9cdb-4ea4738cedc2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:44.886630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:44.888644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:44.889856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:44.889856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:44.889856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:44.889856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:44.889856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:44.889856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:44.889856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:44.889856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:44.889856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 13:02:44.889856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:44.895358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 493/973] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:02:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:44.896198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca2ae2-8ea6-44e2-9cdb-4ea4738cedc2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:44.909579 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:44.944729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e89f6-ca88-44bc-9574-c0e2a4899324 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:44.980608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116be70-e67e-4e06-85ed-4f3002290224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:44.981984 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116be70-e67e-4e06-85ed-4f3002290224 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:44.982142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116be70-e67e-4e06-85ed-4f3002290224 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:44.982334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116be70-e67e-4e06-85ed-4f3002290224 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:44.990069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116be70-e67e-4e06-85ed-4f3002290224 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:44.991612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116be70-e67e-4e06-85ed-4f3002290224 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7116be70-e67e-4e06-85ed-4f3002290224 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:45.005327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:45.011707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:45.013021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:45.013021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:45.013021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:45.013021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:45.013021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:45.013021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:45.013021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:45.013021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:45.013021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 13:02:45.013021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:45.013021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 481/974] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 13:02:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:45.013021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116be70-e67e-4e06-85ed-4f3002290224 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:45.021210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca2ae2-8ea6-44e2-9cdb-4ea4738cedc2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:45.060615 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c286d-0aac-4116-b8d0-72b5779b5cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:45.061153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c286d-0aac-4116-b8d0-72b5779b5cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:45.061428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c286d-0aac-4116-b8d0-72b5779b5cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:45.061696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c286d-0aac-4116-b8d0-72b5779b5cf6 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:45.062329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c286d-0aac-4116-b8d0-72b5779b5cf6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:45.062630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c286d-0aac-4116-b8d0-72b5779b5cf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:45.064016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c286d-0aac-4116-b8d0-72b5779b5cf6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:45.069025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 494/975] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 13:02:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:45.072636 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c286d-0aac-4116-b8d0-72b5779b5cf6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:45.113239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:45.121378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116be70-e67e-4e06-85ed-4f3002290224 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:45.152131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:45.155622 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5931b-db8f-4918-9971-4294318f1673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:45.156229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5931b-db8f-4918-9971-4294318f1673 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:45.156512 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5931b-db8f-4918-9971-4294318f1673 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:45.156777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5931b-db8f-4918-9971-4294318f1673 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:45.157490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5931b-db8f-4918-9971-4294318f1673 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:45.157786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5931b-db8f-4918-9971-4294318f1673 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:45.159232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5931b-db8f-4918-9971-4294318f1673 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:45.177168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c286d-0aac-4116-b8d0-72b5779b5cf6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:45.195980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 482/976] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 9 13:02:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:45.195980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5931b-db8f-4918-9971-4294318f1673 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:45.211259 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12ad50-29b5-49a3-b442-fae1c3f363fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:45.211982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12ad50-29b5-49a3-b442-fae1c3f363fd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:45.212332 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12ad50-29b5-49a3-b442-fae1c3f363fd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:45.212676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12ad50-29b5-49a3-b442-fae1c3f363fd None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:45.213511 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12ad50-29b5-49a3-b442-fae1c3f363fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:45.213893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12ad50-29b5-49a3-b442-fae1c3f363fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:45.216026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12ad50-29b5-49a3-b442-fae1c3f363fd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:45.241814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 495/977] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 9 13:02:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:45.241814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed8c39-231c-4a0f-9544-e52a5402d852 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:45.241814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed8c39-231c-4a0f-9544-e52a5402d852 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:45.241814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed8c39-231c-4a0f-9544-e52a5402d852 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:45.241814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 496/978] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:45.245741 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed8c39-231c-4a0f-9544-e52a5402d852 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:45.318674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5931b-db8f-4918-9971-4294318f1673 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:45.357968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e27b05-0e29-4a4e-a2d5-dcb71ce3647a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:45.358600 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e27b05-0e29-4a4e-a2d5-dcb71ce3647a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:45.358881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e27b05-0e29-4a4e-a2d5-dcb71ce3647a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:45.359186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e27b05-0e29-4a4e-a2d5-dcb71ce3647a None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:45.359838 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e27b05-0e29-4a4e-a2d5-dcb71ce3647a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:45.360143 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e27b05-0e29-4a4e-a2d5-dcb71ce3647a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:45.361137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed8c39-231c-4a0f-9544-e52a5402d852 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:45.361541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e27b05-0e29-4a4e-a2d5-dcb71ce3647a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:45.371008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 483/979] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 13:02:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:45.371008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae22be1-0931-42e6-97e0-d00d3cd9cc69 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:45.371008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae22be1-0931-42e6-97e0-d00d3cd9cc69 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:45.371008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae22be1-0931-42e6-97e0-d00d3cd9cc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:45.391876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fae22be1-0931-42e6-97e0-d00d3cd9cc69 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:45.393414 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fae22be1-0931-42e6-97e0-d00d3cd9cc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:45.400121 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:45.406555 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b3666-6958-4a0c-9ba0-1e7b12d3075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:45.406762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:45.407281 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3666-6958-4a0c-9ba0-1e7b12d3075e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:45.407568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3666-6958-4a0c-9ba0-1e7b12d3075e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:45.407841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3666-6958-4a0c-9ba0-1e7b12d3075e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:45.481368 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 484/980] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:45.482311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae22be1-0931-42e6-97e0-d00d3cd9cc69 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:45.487567 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b3666-6958-4a0c-9ba0-1e7b12d3075e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:45.489119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b3666-6958-4a0c-9ba0-1e7b12d3075e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:45.588920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae22be1-0931-42e6-97e0-d00d3cd9cc69 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:45.601472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 497/981] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:45 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:02:45.607110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b3666-6958-4a0c-9ba0-1e7b12d3075e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:45.631490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a3e9f-f47c-4ef6-84e5-f36ba5bf77b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:45.632106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a3e9f-f47c-4ef6-84e5-f36ba5bf77b2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:45.632385 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a3e9f-f47c-4ef6-84e5-f36ba5bf77b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:45.632656 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a3e9f-f47c-4ef6-84e5-f36ba5bf77b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:45.633365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a3e9f-f47c-4ef6-84e5-f36ba5bf77b2 None admin] RBAC: Adding query filter params (interface=admin, service_id=e44a78cfc2694ab983a953cdd82e55c2, region_id=RegionOne) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:45.633674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a3e9f-f47c-4ef6-84e5-f36ba5bf77b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:45.635037 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a3e9f-f47c-4ef6-84e5-f36ba5bf77b2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:45.640522 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:45.640964 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 485/982] 10.222.0.17 () {56 vars in 1249 bytes} [Fri Sep 9 13:02:45 2022] GET /identity/v3/endpoints?service_id=e44a78cfc2694ab983a953cdd82e55c2&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:45.646219 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a3e9f-f47c-4ef6-84e5-f36ba5bf77b2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:45.658092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:45.717035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b3666-6958-4a0c-9ba0-1e7b12d3075e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:45.752805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a3e9f-f47c-4ef6-84e5-f36ba5bf77b2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:45.755508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef9186-f787-4e5d-b47b-48f698dc8fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:45.756150 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef9186-f787-4e5d-b47b-48f698dc8fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:45.756432 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef9186-f787-4e5d-b47b-48f698dc8fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:45.756707 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef9186-f787-4e5d-b47b-48f698dc8fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:45.767212 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef9186-f787-4e5d-b47b-48f698dc8fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:45.768659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef9186-f787-4e5d-b47b-48f698dc8fe6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-14ef9186-f787-4e5d-b47b-48f698dc8fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:45.783152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:45.784105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:45.785123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:45.785123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:45.785123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:45.786284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cbca3-4621-4d58-b140-1502f3f7f1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:45.786893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cbca3-4621-4d58-b140-1502f3f7f1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:45.787213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cbca3-4621-4d58-b140-1502f3f7f1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:45.787486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cbca3-4621-4d58-b140-1502f3f7f1e8 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:45.788087 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cbca3-4621-4d58-b140-1502f3f7f1e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:45.788857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 498/983] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:45.789452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cbca3-4621-4d58-b140-1502f3f7f1e8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:45.789631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef9186-f787-4e5d-b47b-48f698dc8fe6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:45.799169 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 486/984] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 13:02:45 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:02:45.799169 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cbca3-4621-4d58-b140-1502f3f7f1e8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:45.883791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:45.895201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:45.913965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cbca3-4621-4d58-b140-1502f3f7f1e8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:45.914225 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef9186-f787-4e5d-b47b-48f698dc8fe6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:45.953139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fc641-d1e6-4814-a73f-09af1c7e4138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:45.953775 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fc641-d1e6-4814-a73f-09af1c7e4138 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:45.954064 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fc641-d1e6-4814-a73f-09af1c7e4138 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:45.954337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fc641-d1e6-4814-a73f-09af1c7e4138 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:45.955000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fc641-d1e6-4814-a73f-09af1c7e4138 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:45.956170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a47f77-5678-4318-9df4-bcd735da684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:45.956511 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fc641-d1e6-4814-a73f-09af1c7e4138 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:45.957121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a47f77-5678-4318-9df4-bcd735da684d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:45.957575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a47f77-5678-4318-9df4-bcd735da684d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:45.957993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a47f77-5678-4318-9df4-bcd735da684d None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:45.958797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a47f77-5678-4318-9df4-bcd735da684d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:45.959284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a47f77-5678-4318-9df4-bcd735da684d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:45.961059 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a47f77-5678-4318-9df4-bcd735da684d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:45.972713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 499/985] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:02:45 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:45.978262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 487/986] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:45.979720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fc641-d1e6-4814-a73f-09af1c7e4138 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:46.103586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fc641-d1e6-4814-a73f-09af1c7e4138 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:46.141650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826e16f-598c-4173-b3c1-2d45ef2e434b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:46.142606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826e16f-598c-4173-b3c1-2d45ef2e434b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:46.143095 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826e16f-598c-4173-b3c1-2d45ef2e434b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:46.143553 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826e16f-598c-4173-b3c1-2d45ef2e434b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:46.151189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826e16f-598c-4173-b3c1-2d45ef2e434b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:46.152947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826e16f-598c-4173-b3c1-2d45ef2e434b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:46.156198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a826e16f-598c-4173-b3c1-2d45ef2e434b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:46.165766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:46.166684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:46.167735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:46.167735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:46.167735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:46.167735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:46.167735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:46.167735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:46.167735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:46.167735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:46.167735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 500/987] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:46.172914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a47f77-5678-4318-9df4-bcd735da684d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:46.279477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a47f77-5678-4318-9df4-bcd735da684d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:46.315162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1b0bf-2659-4ce5-b2e9-7197b97a8f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:46.315783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1b0bf-2659-4ce5-b2e9-7197b97a8f7e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:46.316061 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1b0bf-2659-4ce5-b2e9-7197b97a8f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:46.316329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1b0bf-2659-4ce5-b2e9-7197b97a8f7e None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:46.317023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1b0bf-2659-4ce5-b2e9-7197b97a8f7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:46.317332 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1b0bf-2659-4ce5-b2e9-7197b97a8f7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:46.318752 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1b0bf-2659-4ce5-b2e9-7197b97a8f7e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:46.320621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:46.337316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 488/988] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:46.338703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826e16f-598c-4173-b3c1-2d45ef2e434b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:46.445530 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826e16f-598c-4173-b3c1-2d45ef2e434b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:46.481920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39020f-4c4d-4ea8-8466-3d2908f3cac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:46.482532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39020f-4c4d-4ea8-8466-3d2908f3cac1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:46.482809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39020f-4c4d-4ea8-8466-3d2908f3cac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:46.483107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39020f-4c4d-4ea8-8466-3d2908f3cac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:46.483839 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39020f-4c4d-4ea8-8466-3d2908f3cac1 None admin] RBAC: Adding query filter params (role.id=bd6892add1ee40ac8720e915e153c34f, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=8d343a3db21c47e1a85a739a55559c97) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:46.484147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39020f-4c4d-4ea8-8466-3d2908f3cac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:46.485573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39020f-4c4d-4ea8-8466-3d2908f3cac1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:46.491817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 501/989] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:46 2022] GET /identity/v3/role_assignments?user.id=8d343a3db21c47e1a85a739a55559c97&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=bd6892add1ee40ac8720e915e153c34f => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:48.001977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22360d-0eed-41fe-a3fc-a743097d81a9 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:48.002501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22360d-0eed-41fe-a3fc-a743097d81a9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:48.002501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22360d-0eed-41fe-a3fc-a743097d81a9 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:48.004076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 489/990] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:48.009239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af77e75-5c2f-4804-adc5-7de599d4d05f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:48.009678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af77e75-5c2f-4804-adc5-7de599d4d05f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:48.009824 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af77e75-5c2f-4804-adc5-7de599d4d05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:48.029451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5af77e75-5c2f-4804-adc5-7de599d4d05f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:48.030671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5af77e75-5c2f-4804-adc5-7de599d4d05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:48.096381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:48.106490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 502/991] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:48.165061 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed068d-2f99-4d6a-bba6-8c2fc69de07b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:48.165370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed068d-2f99-4d6a-bba6-8c2fc69de07b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:48.165613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed068d-2f99-4d6a-bba6-8c2fc69de07b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:48.167109 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 490/992] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:48.172746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f8316-5180-411e-b040-1ea97204f1ef None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:48.173035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f8316-5180-411e-b040-1ea97204f1ef None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:48.173313 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f8316-5180-411e-b040-1ea97204f1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:48.191665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-879f8316-5180-411e-b040-1ea97204f1ef None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:48.192868 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-879f8316-5180-411e-b040-1ea97204f1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:48.262683 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 503/993] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:48.374092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed068d-2f99-4d6a-bba6-8c2fc69de07b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:48.476179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed068d-2f99-4d6a-bba6-8c2fc69de07b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:48.510206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1f770-7d3a-4b5d-b248-2f47800c0ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:48.510839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1f770-7d3a-4b5d-b248-2f47800c0ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:48.511140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1f770-7d3a-4b5d-b248-2f47800c0ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:48.511415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1f770-7d3a-4b5d-b248-2f47800c0ec6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:48.512085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1f770-7d3a-4b5d-b248-2f47800c0ec6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:48.513459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1f770-7d3a-4b5d-b248-2f47800c0ec6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c9c1f770-7d3a-4b5d-b248-2f47800c0ec6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:48.522029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:48.523014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 13:02:48.524190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:48.527875 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 491/994] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 9 13:02:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:48.530071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f8316-5180-411e-b040-1ea97204f1ef None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:48.532474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1f770-7d3a-4b5d-b248-2f47800c0ec6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:48.597714 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:48.645684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1f770-7d3a-4b5d-b248-2f47800c0ec6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:48.650921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f8316-5180-411e-b040-1ea97204f1ef None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:48.678145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:48.686505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6e791-e678-4ca5-92f8-3bc134a4c911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:48.687261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6e791-e678-4ca5-92f8-3bc134a4c911 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:48.687544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6e791-e678-4ca5-92f8-3bc134a4c911 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:48.687816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6e791-e678-4ca5-92f8-3bc134a4c911 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:48.688400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6e791-e678-4ca5-92f8-3bc134a4c911 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:48.689572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f36d7d-7bc7-40ce-b340-22540696019c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:48.689775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6e791-e678-4ca5-92f8-3bc134a4c911 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:48.690180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f36d7d-7bc7-40ce-b340-22540696019c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:48.690455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f36d7d-7bc7-40ce-b340-22540696019c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:48.690722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f36d7d-7bc7-40ce-b340-22540696019c None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:48.691497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f36d7d-7bc7-40ce-b340-22540696019c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:48.691675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f36d7d-7bc7-40ce-b340-22540696019c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:48.693013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f36d7d-7bc7-40ce-b340-22540696019c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:48.697620 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 504/995] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 13:02:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4dd6e791-e678-4ca5-92f8-3bc134a4c911 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:48.698538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:48.700202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 13:02:48.701072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:48.701815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12871b55-ed80-482d-a680-2223cf35f7b4 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:48.701815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12871b55-ed80-482d-a680-2223cf35f7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:48.701815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12871b55-ed80-482d-a680-2223cf35f7b4 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:48.702003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 492/996] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:48.702071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 505/997] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:48.703681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6e791-e678-4ca5-92f8-3bc134a4c911 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:48.708504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ff8cd-cc5f-4e07-9a01-bf76d011d1f3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:48.708785 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ff8cd-cc5f-4e07-9a01-bf76d011d1f3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:48.709061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ff8cd-cc5f-4e07-9a01-bf76d011d1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:48.726738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-647ff8cd-cc5f-4e07-9a01-bf76d011d1f3 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:48.727991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-647ff8cd-cc5f-4e07-9a01-bf76d011d1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:48.797737 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 506/998] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:48.798503 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ff8cd-cc5f-4e07-9a01-bf76d011d1f3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:48.806327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6e791-e678-4ca5-92f8-3bc134a4c911 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:48.832565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:48.844988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03bf7b-6d5e-4e05-b5fb-be6ef200f188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:48.845599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03bf7b-6d5e-4e05-b5fb-be6ef200f188 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:48.845868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03bf7b-6d5e-4e05-b5fb-be6ef200f188 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:48.846136 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03bf7b-6d5e-4e05-b5fb-be6ef200f188 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:48.846763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03bf7b-6d5e-4e05-b5fb-be6ef200f188 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:48.847074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03bf7b-6d5e-4e05-b5fb-be6ef200f188 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:48.848710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03bf7b-6d5e-4e05-b5fb-be6ef200f188 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:48.858458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 493/999] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 9 13:02:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:48.859253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03bf7b-6d5e-4e05-b5fb-be6ef200f188 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:48.909651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ff8cd-cc5f-4e07-9a01-bf76d011d1f3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:48.929263 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:48.948015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6cb83-7a33-4341-8fa7-df89fe7ec145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:48.948717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6cb83-7a33-4341-8fa7-df89fe7ec145 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:48.949002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6cb83-7a33-4341-8fa7-df89fe7ec145 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:48.949800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6cb83-7a33-4341-8fa7-df89fe7ec145 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:48.950687 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6cb83-7a33-4341-8fa7-df89fe7ec145 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:48.951116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6cb83-7a33-4341-8fa7-df89fe7ec145 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:48.952578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6cb83-7a33-4341-8fa7-df89fe7ec145 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:48.957428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 507/1000] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 13:02:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:48.962951 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6cb83-7a33-4341-8fa7-df89fe7ec145 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:48.976069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03bf7b-6d5e-4e05-b5fb-be6ef200f188 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:49.012139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d203a99-fd18-4960-a80c-0910cf4d877d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:49.012900 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d203a99-fd18-4960-a80c-0910cf4d877d None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:49.013180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d203a99-fd18-4960-a80c-0910cf4d877d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:49.013451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d203a99-fd18-4960-a80c-0910cf4d877d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:49.014119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d203a99-fd18-4960-a80c-0910cf4d877d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:49.015514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d203a99-fd18-4960-a80c-0910cf4d877d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:49.043357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 494/1001] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:02:49.069174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d203a99-fd18-4960-a80c-0910cf4d877d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:49.086291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6cb83-7a33-4341-8fa7-df89fe7ec145 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:49.124224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:49.134700 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7eb99-3f66-4f65-bab1-afe74bbc4381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:49.135416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7eb99-3f66-4f65-bab1-afe74bbc4381 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:49.135640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7eb99-3f66-4f65-bab1-afe74bbc4381 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:49.135916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7eb99-3f66-4f65-bab1-afe74bbc4381 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:49.136584 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7eb99-3f66-4f65-bab1-afe74bbc4381 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:49.136897 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7eb99-3f66-4f65-bab1-afe74bbc4381 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:49.138398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7eb99-3f66-4f65-bab1-afe74bbc4381 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:49.148867 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 508/1002] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 13:02:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:49.150001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7eb99-3f66-4f65-bab1-afe74bbc4381 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:49.196399 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d203a99-fd18-4960-a80c-0910cf4d877d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:49.198777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:49.239268 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5f14b-2245-4aff-83d4-568ada3d6b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:49.239935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5f14b-2245-4aff-83d4-568ada3d6b26 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:49.240516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5f14b-2245-4aff-83d4-568ada3d6b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:49.240797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5f14b-2245-4aff-83d4-568ada3d6b26 None admin] PATH_INFO: `/v3/services/image` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:49.241682 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5f14b-2245-4aff-83d4-568ada3d6b26 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:49.243306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5f14b-2245-4aff-83d4-568ada3d6b26 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9cd5f14b-2245-4aff-83d4-568ada3d6b26 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:49.253379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:49.254615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:49.255593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 495/1003] 10.222.0.17 () {56 vars in 1105 bytes} [Fri Sep 9 13:02:49 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:49.260458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5f14b-2245-4aff-83d4-568ada3d6b26 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:49.266963 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7eb99-3f66-4f65-bab1-afe74bbc4381 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:49.302916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3138fc-2ebc-4d29-97d4-8190ad89d6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:49.303566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3138fc-2ebc-4d29-97d4-8190ad89d6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:49.303849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3138fc-2ebc-4d29-97d4-8190ad89d6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:49.304124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3138fc-2ebc-4d29-97d4-8190ad89d6c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:49.304829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3138fc-2ebc-4d29-97d4-8190ad89d6c4 None admin] RBAC: Adding query filter params (interface=public, service_id=76b854e283fe448ca5fb2c597705be3c, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:49.305145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3138fc-2ebc-4d29-97d4-8190ad89d6c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:49.306519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3138fc-2ebc-4d29-97d4-8190ad89d6c4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:49.317341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 509/1004] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 13:02:49 2022] GET /identity/v3/endpoints?service_id=76b854e283fe448ca5fb2c597705be3c&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:49.318401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3138fc-2ebc-4d29-97d4-8190ad89d6c4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:49.373428 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5f14b-2245-4aff-83d4-568ada3d6b26 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:49.385357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:49.414834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11170c6-243a-4e4b-b6ca-760cc3292b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:49.415739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11170c6-243a-4e4b-b6ca-760cc3292b4e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:49.416186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11170c6-243a-4e4b-b6ca-760cc3292b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:49.416459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11170c6-243a-4e4b-b6ca-760cc3292b4e None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:49.417097 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11170c6-243a-4e4b-b6ca-760cc3292b4e None admin] RBAC: Adding query filter params (name=image) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:49.417402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11170c6-243a-4e4b-b6ca-760cc3292b4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:49.419362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11170c6-243a-4e4b-b6ca-760cc3292b4e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:49.428749 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 496/1005] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:49.429784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11170c6-243a-4e4b-b6ca-760cc3292b4e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:49.437207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3138fc-2ebc-4d29-97d4-8190ad89d6c4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:49.446027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:49.478116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3a6f0-a58d-462a-b7d1-6af0c5e4b1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:49.478738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3a6f0-a58d-462a-b7d1-6af0c5e4b1dd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:49.479018 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3a6f0-a58d-462a-b7d1-6af0c5e4b1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:49.479378 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3a6f0-a58d-462a-b7d1-6af0c5e4b1dd None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:49.479988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3a6f0-a58d-462a-b7d1-6af0c5e4b1dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:49.481606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3a6f0-a58d-462a-b7d1-6af0c5e4b1dd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:49.499088 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 510/1006] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 13:02:49 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:02:49.550638 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11170c6-243a-4e4b-b6ca-760cc3292b4e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:49.588386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983452c3-029a-4dc0-aa01-85aeb7ef51f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:49.589010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983452c3-029a-4dc0-aa01-85aeb7ef51f4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:49.589290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983452c3-029a-4dc0-aa01-85aeb7ef51f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:49.589560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983452c3-029a-4dc0-aa01-85aeb7ef51f4 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:49.590786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983452c3-029a-4dc0-aa01-85aeb7ef51f4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:49.591126 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983452c3-029a-4dc0-aa01-85aeb7ef51f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:49.592734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983452c3-029a-4dc0-aa01-85aeb7ef51f4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:49.600192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 497/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:51.265966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cc835-f56d-4637-ae1e-f5edd3fd32fb None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.266533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cc835-f56d-4637-ae1e-f5edd3fd32fb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.266533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cc835-f56d-4637-ae1e-f5edd3fd32fb None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.267924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 511/1008] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:51.273381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cc282-76be-4fba-948b-c41137ef81d9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.273797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cc282-76be-4fba-948b-c41137ef81d9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.273939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cc282-76be-4fba-948b-c41137ef81d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.293281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5a2cc282-76be-4fba-948b-c41137ef81d9 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:51.294475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5a2cc282-76be-4fba-948b-c41137ef81d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:51.326599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:51.367124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 498/1009] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:51 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:51.636748 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cc835-f56d-4637-ae1e-f5edd3fd32fb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:51.684235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e9418-a7e7-46a0-afbf-dcfa3bd9cada None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.684807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e9418-a7e7-46a0-afbf-dcfa3bd9cada None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.685302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e9418-a7e7-46a0-afbf-dcfa3bd9cada None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.686908 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 499/1010] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:51.692319 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3cc33-c059-4efc-85b4-c25e00d57280 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.692616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3cc33-c059-4efc-85b4-c25e00d57280 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.692894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3cc33-c059-4efc-85b4-c25e00d57280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.711263 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-97d3cc33-c059-4efc-85b4-c25e00d57280 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:51.712505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-97d3cc33-c059-4efc-85b4-c25e00d57280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:51.755164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cc835-f56d-4637-ae1e-f5edd3fd32fb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:51.784286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 500/1011] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:51 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:51.788463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd82ba-585a-4ebb-a46e-e99143356e25 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.788748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd82ba-585a-4ebb-a46e-e99143356e25 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.789028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd82ba-585a-4ebb-a46e-e99143356e25 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.790299 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 501/1012] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:51.795890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21809e4-c513-48ad-bdc3-9c4a05f07333 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.796384 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21809e4-c513-48ad-bdc3-9c4a05f07333 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.796661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21809e4-c513-48ad-bdc3-9c4a05f07333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.801645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a12c33-c521-40b2-afc7-8dee73e188d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:51.802277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12c33-c521-40b2-afc7-8dee73e188d3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.802675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12c33-c521-40b2-afc7-8dee73e188d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.802943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12c33-c521-40b2-afc7-8dee73e188d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.804078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a12c33-c521-40b2-afc7-8dee73e188d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:51.805494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a12c33-c521-40b2-afc7-8dee73e188d3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:51.807218 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 512/1013] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:51.813346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a12c33-c521-40b2-afc7-8dee73e188d3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:51.815102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f21809e4-c513-48ad-bdc3-9c4a05f07333 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:51.816301 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f21809e4-c513-48ad-bdc3-9c4a05f07333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:51.855547 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:51.893800 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 502/1014] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:51 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:51.918023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a12c33-c521-40b2-afc7-8dee73e188d3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:51.954374 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75c0c0-3bcd-4828-af0f-436f18b69589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:51.954986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75c0c0-3bcd-4828-af0f-436f18b69589 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.955297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75c0c0-3bcd-4828-af0f-436f18b69589 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.955578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75c0c0-3bcd-4828-af0f-436f18b69589 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.956212 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75c0c0-3bcd-4828-af0f-436f18b69589 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:51.957632 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75c0c0-3bcd-4828-af0f-436f18b69589 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:51.991076 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 513/1015] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:02:51 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:02:52.061591 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21809e4-c513-48ad-bdc3-9c4a05f07333 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:52.091884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:52.169202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21809e4-c513-48ad-bdc3-9c4a05f07333 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:52.172111 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75c0c0-3bcd-4828-af0f-436f18b69589 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:52.208129 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f8f8d-c782-40b6-afea-cefa89cb42ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:52.208762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f8f8d-c782-40b6-afea-cefa89cb42ff None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.209104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f8f8d-c782-40b6-afea-cefa89cb42ff None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.209330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f8f8d-c782-40b6-afea-cefa89cb42ff None admin] PATH_INFO: `/v3/services/compute` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.209916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f8f8d-c782-40b6-afea-cefa89cb42ff None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:52.211553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f8f8d-c782-40b6-afea-cefa89cb42ff None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-445f8f8d-c782-40b6-afea-cefa89cb42ff None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:52.220906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:52.221917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:52.222795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 503/1016] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:52.227922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f8f8d-c782-40b6-afea-cefa89cb42ff None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:52.286732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:52.298841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75c0c0-3bcd-4828-af0f-436f18b69589 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:52.332840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:52.338293 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebcd84-a2f8-4b2f-92bc-f775bef772a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:52.339018 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebcd84-a2f8-4b2f-92bc-f775bef772a0 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.339391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebcd84-a2f8-4b2f-92bc-f775bef772a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.339774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebcd84-a2f8-4b2f-92bc-f775bef772a0 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.340629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebcd84-a2f8-4b2f-92bc-f775bef772a0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:52.341996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebcd84-a2f8-4b2f-92bc-f775bef772a0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:52.342677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f8f8d-c782-40b6-afea-cefa89cb42ff None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:52.359993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 514/1017] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:02:52 2022] POST /identity/v3/services => generated 242 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:02:52.391912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd12d9-6d0b-43f1-a166-abd71b78ce98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:52.392590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd12d9-6d0b-43f1-a166-abd71b78ce98 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.392880 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd12d9-6d0b-43f1-a166-abd71b78ce98 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.393157 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd12d9-6d0b-43f1-a166-abd71b78ce98 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.393805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd12d9-6d0b-43f1-a166-abd71b78ce98 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:52.394114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd12d9-6d0b-43f1-a166-abd71b78ce98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:52.395625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd12d9-6d0b-43f1-a166-abd71b78ce98 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:52.407123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 504/1018] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 13:02:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:52.408469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebcd84-a2f8-4b2f-92bc-f775bef772a0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:52.523427 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebcd84-a2f8-4b2f-92bc-f775bef772a0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:52.564050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a69b2-72d9-46d2-9300-5e969fce3a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:52.564981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a69b2-72d9-46d2-9300-5e969fce3a83 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.565266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a69b2-72d9-46d2-9300-5e969fce3a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.565543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a69b2-72d9-46d2-9300-5e969fce3a83 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.566178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a69b2-72d9-46d2-9300-5e969fce3a83 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:52.566484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a69b2-72d9-46d2-9300-5e969fce3a83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:52.567895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a69b2-72d9-46d2-9300-5e969fce3a83 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:52.573464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 515/1019] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 9 13:02:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:52.580347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd12d9-6d0b-43f1-a166-abd71b78ce98 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:52.690797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd12d9-6d0b-43f1-a166-abd71b78ce98 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:52.725808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864b85a-1701-48f9-908a-39aeb71b86ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:52.726423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864b85a-1701-48f9-908a-39aeb71b86ed None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.726704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864b85a-1701-48f9-908a-39aeb71b86ed None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.726979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864b85a-1701-48f9-908a-39aeb71b86ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.727772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864b85a-1701-48f9-908a-39aeb71b86ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:52.729157 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864b85a-1701-48f9-908a-39aeb71b86ed None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:52.755088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 505/1020] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:02:54.234348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb47b72-b06e-4408-9057-764b51323285 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:54.234870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb47b72-b06e-4408-9057-764b51323285 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:54.234870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb47b72-b06e-4408-9057-764b51323285 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:54.236306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 516/1021] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:54.241596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7ed73-3736-4a9c-98ef-4cfeb749e366 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:54.242011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7ed73-3736-4a9c-98ef-4cfeb749e366 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:54.242181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7ed73-3736-4a9c-98ef-4cfeb749e366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:54.260518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c9f7ed73-3736-4a9c-98ef-4cfeb749e366 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:54.262263 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c9f7ed73-3736-4a9c-98ef-4cfeb749e366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:54.264223 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:54.338206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 506/1022] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:54.563995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91293f6-27cd-4173-9805-75ccbce40918 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:54.564189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91293f6-27cd-4173-9805-75ccbce40918 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:54.564455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91293f6-27cd-4173-9805-75ccbce40918 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:54.565935 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 517/1023] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:54.571083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df83e8-9937-4ae5-bc43-66725f30006c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:54.571378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df83e8-9937-4ae5-bc43-66725f30006c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:54.571657 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df83e8-9937-4ae5-bc43-66725f30006c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:54.590456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-21df83e8-9937-4ae5-bc43-66725f30006c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:54.593719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-21df83e8-9937-4ae5-bc43-66725f30006c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:54.612203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91293f6-27cd-4173-9805-75ccbce40918 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:54.629180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:54.669861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 507/1024] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:02:54.723215 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91293f6-27cd-4173-9805-75ccbce40918 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:54.757440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75cbb6-e6bc-406d-8a23-72a01ab61735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:54.758073 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75cbb6-e6bc-406d-8a23-72a01ab61735 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:54.758352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75cbb6-e6bc-406d-8a23-72a01ab61735 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:54.758626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75cbb6-e6bc-406d-8a23-72a01ab61735 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:54.759752 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75cbb6-e6bc-406d-8a23-72a01ab61735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:54.761777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75cbb6-e6bc-406d-8a23-72a01ab61735 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:54.763599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 518/1025] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:54.769162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df83e8-9937-4ae5-bc43-66725f30006c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:54.832252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:54.871699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df83e8-9937-4ae5-bc43-66725f30006c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:54.910038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3de80-9c6d-4bed-bcad-8346d2ed6f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:54.910684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3de80-9c6d-4bed-bcad-8346d2ed6f6b None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:54.910962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3de80-9c6d-4bed-bcad-8346d2ed6f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:54.911267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3de80-9c6d-4bed-bcad-8346d2ed6f6b None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:54.911948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3de80-9c6d-4bed-bcad-8346d2ed6f6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:54.913423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3de80-9c6d-4bed-bcad-8346d2ed6f6b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:54.936350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 508/1026] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:02:54 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:54.939563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75cbb6-e6bc-406d-8a23-72a01ab61735 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:55.017344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9551850-8558-43f8-b7d6-2b24429ec223 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:55.017729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9551850-8558-43f8-b7d6-2b24429ec223 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:55.018069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9551850-8558-43f8-b7d6-2b24429ec223 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:55.019554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 509/1027] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:02:55.025280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94facb41-72c9-4d64-b439-d5b14c3a4dee None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:55.025874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94facb41-72c9-4d64-b439-d5b14c3a4dee None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:55.026304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94facb41-72c9-4d64-b439-d5b14c3a4dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:55.041458 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:55.046625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-94facb41-72c9-4d64-b439-d5b14c3a4dee None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:55.047873 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-94facb41-72c9-4d64-b439-d5b14c3a4dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:55.062858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75cbb6-e6bc-406d-8a23-72a01ab61735 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:55.102817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6acd5-8fbf-43cd-95dd-7a45a257fbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:55.103746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6acd5-8fbf-43cd-95dd-7a45a257fbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:55.104206 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6acd5-8fbf-43cd-95dd-7a45a257fbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:55.104653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6acd5-8fbf-43cd-95dd-7a45a257fbb0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:55.105415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6acd5-8fbf-43cd-95dd-7a45a257fbb0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:55.107003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6acd5-8fbf-43cd-95dd-7a45a257fbb0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7ff6acd5-8fbf-43cd-95dd-7a45a257fbb0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:55.117369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:55.118628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:55.119705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 519/1028] 10.222.0.17 () {56 vars in 1105 bytes} [Fri Sep 9 13:02:54 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:55.124983 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6acd5-8fbf-43cd-95dd-7a45a257fbb0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:55.125125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 510/1029] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:55.235391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6acd5-8fbf-43cd-95dd-7a45a257fbb0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:55.270490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25bd96-2342-4473-a44b-f8af1a465881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:55.271128 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25bd96-2342-4473-a44b-f8af1a465881 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:55.271408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25bd96-2342-4473-a44b-f8af1a465881 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:55.271676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25bd96-2342-4473-a44b-f8af1a465881 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:55.272296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25bd96-2342-4473-a44b-f8af1a465881 None admin] RBAC: Adding query filter params (name=image) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:55.272594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25bd96-2342-4473-a44b-f8af1a465881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:55.273940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25bd96-2342-4473-a44b-f8af1a465881 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:55.279653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 520/1030] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:55.285136 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94facb41-72c9-4d64-b439-d5b14c3a4dee None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:55.337159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:55.390301 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94facb41-72c9-4d64-b439-d5b14c3a4dee None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:55.395715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25bd96-2342-4473-a44b-f8af1a465881 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:55.408090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:55.427981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856319ba-9e9e-4923-b6ba-cbe3d0a74cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:55.428599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856319ba-9e9e-4923-b6ba-cbe3d0a74cce None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:55.428878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856319ba-9e9e-4923-b6ba-cbe3d0a74cce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:55.429153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856319ba-9e9e-4923-b6ba-cbe3d0a74cce None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:55.429787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856319ba-9e9e-4923-b6ba-cbe3d0a74cce None admin] RBAC: Adding query filter params (type=image) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:55.430095 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856319ba-9e9e-4923-b6ba-cbe3d0a74cce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:55.431514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856319ba-9e9e-4923-b6ba-cbe3d0a74cce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:55.437632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 511/1031] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:55.443204 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856319ba-9e9e-4923-b6ba-cbe3d0a74cce None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:55.510016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25bd96-2342-4473-a44b-f8af1a465881 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:55.547024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7eeaa-4f62-4bc2-a95a-0f1f3414cfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:55.547694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7eeaa-4f62-4bc2-a95a-0f1f3414cfa1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:55.547974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7eeaa-4f62-4bc2-a95a-0f1f3414cfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:55.548247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7eeaa-4f62-4bc2-a95a-0f1f3414cfa1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:55.549055 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856319ba-9e9e-4923-b6ba-cbe3d0a74cce None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:55.558967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d7eeaa-4f62-4bc2-a95a-0f1f3414cfa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-b3d7eeaa-4f62-4bc2-a95a-0f1f3414cfa1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:55.563770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:55.564717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:55.565564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:55.565564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:55.565564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:55.565564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:55.565564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:55.565564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:55.565564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:55.565564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:55.565564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:55.565564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:55.565564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 13:02:55.565564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:55.565564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 521/1032] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 9 13:02:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:55.568482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:55.571169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7eeaa-4f62-4bc2-a95a-0f1f3414cfa1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:55.595602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a448b7-42a9-46a3-bd1f-52b8fcf0462d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:55.596225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a448b7-42a9-46a3-bd1f-52b8fcf0462d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:55.596505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a448b7-42a9-46a3-bd1f-52b8fcf0462d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:55.597213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a448b7-42a9-46a3-bd1f-52b8fcf0462d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:55.597491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a448b7-42a9-46a3-bd1f-52b8fcf0462d None admin] RBAC: Adding query filter params (interface=public, service_id=819499573c36472d80c25cac118f2550, region_id=RegionOne) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:55.597803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a448b7-42a9-46a3-bd1f-52b8fcf0462d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:55.599243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a448b7-42a9-46a3-bd1f-52b8fcf0462d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:55.605508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 512/1033] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 13:02:55 2022] GET /identity/v3/endpoints?service_id=819499573c36472d80c25cac118f2550&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:55.611292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a448b7-42a9-46a3-bd1f-52b8fcf0462d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:55.654059 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:55.687729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7eeaa-4f62-4bc2-a95a-0f1f3414cfa1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:55.718013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a448b7-42a9-46a3-bd1f-52b8fcf0462d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:55.723698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808164b9-411e-41b0-b04c-4aa8bde3c076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:55.724556 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808164b9-411e-41b0-b04c-4aa8bde3c076 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:55.725036 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808164b9-411e-41b0-b04c-4aa8bde3c076 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:55.725491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808164b9-411e-41b0-b04c-4aa8bde3c076 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:55.726306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808164b9-411e-41b0-b04c-4aa8bde3c076 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:55.726761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808164b9-411e-41b0-b04c-4aa8bde3c076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:55.728315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808164b9-411e-41b0-b04c-4aa8bde3c076 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:55.737537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 522/1034] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 13:02:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:55.743312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808164b9-411e-41b0-b04c-4aa8bde3c076 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:55.757109 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a864c-2cb9-42d8-a183-6552a4f767d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:55.757715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a864c-2cb9-42d8-a183-6552a4f767d7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:55.757993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a864c-2cb9-42d8-a183-6552a4f767d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:55.758257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a864c-2cb9-42d8-a183-6552a4f767d7 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:55.758856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a864c-2cb9-42d8-a183-6552a4f767d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:55.760256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a864c-2cb9-42d8-a183-6552a4f767d7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:55.779100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 513/1035] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 13:02:55 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:02:55.874520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808164b9-411e-41b0-b04c-4aa8bde3c076 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:55.906487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:55.912967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d612c6-1d9a-45b1-aa3e-700b941d820a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:55.913625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d612c6-1d9a-45b1-aa3e-700b941d820a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:55.913900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d612c6-1d9a-45b1-aa3e-700b941d820a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:55.914171 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d612c6-1d9a-45b1-aa3e-700b941d820a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:55.926864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d612c6-1d9a-45b1-aa3e-700b941d820a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:55.928385 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d612c6-1d9a-45b1-aa3e-700b941d820a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-82d612c6-1d9a-45b1-aa3e-700b941d820a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:55.944352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:55.945413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:55.946372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:55.946372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:55.946372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:55.946549 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 523/1036] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:55.952842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a864c-2cb9-42d8-a183-6552a4f767d7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:55.999822 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:56.060545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a864c-2cb9-42d8-a183-6552a4f767d7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:56.095043 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92567df-87e0-4d5f-8da0-b26114aa8fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:56.095672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92567df-87e0-4d5f-8da0-b26114aa8fef None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:56.095946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92567df-87e0-4d5f-8da0-b26114aa8fef None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:56.096213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92567df-87e0-4d5f-8da0-b26114aa8fef None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:56.096844 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92567df-87e0-4d5f-8da0-b26114aa8fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:56.097142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92567df-87e0-4d5f-8da0-b26114aa8fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:56.098492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92567df-87e0-4d5f-8da0-b26114aa8fef None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:56.114546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 514/1037] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:56.115807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d612c6-1d9a-45b1-aa3e-700b941d820a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:56.221099 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d612c6-1d9a-45b1-aa3e-700b941d820a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:56.256800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f13e33-ab44-4851-9fc6-2a8c13acb969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:56.257459 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f13e33-ab44-4851-9fc6-2a8c13acb969 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:56.257699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f13e33-ab44-4851-9fc6-2a8c13acb969 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:56.257951 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f13e33-ab44-4851-9fc6-2a8c13acb969 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:56.267772 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f13e33-ab44-4851-9fc6-2a8c13acb969 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:56.269210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f13e33-ab44-4851-9fc6-2a8c13acb969 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-50f13e33-ab44-4851-9fc6-2a8c13acb969 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:02:56.280987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:56.281941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:56.282924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:56.282924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:56.282924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:02:56.282924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:02:56.282924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:02:56.282924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:02:56.282924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:02:56.282924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:02:56.282924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:02:56.282924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:56.282924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 524/1038] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 13:02:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:56.289873 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92567df-87e0-4d5f-8da0-b26114aa8fef None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:56.394438 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92567df-87e0-4d5f-8da0-b26114aa8fef None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:56.395960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:56.430036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7609762-3498-406b-bc23-ac30d3e8cb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:56.430640 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609762-3498-406b-bc23-ac30d3e8cb29 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:56.430911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609762-3498-406b-bc23-ac30d3e8cb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:56.431210 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609762-3498-406b-bc23-ac30d3e8cb29 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:56.431922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7609762-3498-406b-bc23-ac30d3e8cb29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:56.432296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7609762-3498-406b-bc23-ac30d3e8cb29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:56.434648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7609762-3498-406b-bc23-ac30d3e8cb29 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:56.456476 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 515/1039] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 13:02:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:56.457915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f13e33-ab44-4851-9fc6-2a8c13acb969 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:56.506030 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:56.562226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f13e33-ab44-4851-9fc6-2a8c13acb969 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:56.598096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a279502-2601-4dee-b7a1-924f8c1f4e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:56.598715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a279502-2601-4dee-b7a1-924f8c1f4e7c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:56.598994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a279502-2601-4dee-b7a1-924f8c1f4e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:56.599297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a279502-2601-4dee-b7a1-924f8c1f4e7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:56.673916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a279502-2601-4dee-b7a1-924f8c1f4e7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:56.675434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a279502-2601-4dee-b7a1-924f8c1f4e7c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:56.721088 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:56.792151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 525/1040] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:56 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 13:02:56.793341 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7609762-3498-406b-bc23-ac30d3e8cb29 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:56.893969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7609762-3498-406b-bc23-ac30d3e8cb29 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:56.927786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b2dc0-a16c-412e-8a57-40cd1d3a619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:56.928420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b2dc0-a16c-412e-8a57-40cd1d3a619e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:56.928703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b2dc0-a16c-412e-8a57-40cd1d3a619e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:56.928978 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b2dc0-a16c-412e-8a57-40cd1d3a619e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:56.939795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b2dc0-a16c-412e-8a57-40cd1d3a619e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:56.941239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b2dc0-a16c-412e-8a57-40cd1d3a619e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bd2b2dc0-a16c-412e-8a57-40cd1d3a619e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:02:56.955790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:56.956773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:02:56.957664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:02:56.957664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:02:56.957664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:56.957664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 516/1041] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:56.963558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a279502-2601-4dee-b7a1-924f8c1f4e7c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:57.071983 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a279502-2601-4dee-b7a1-924f8c1f4e7c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:57.107179 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e045606-b296-4ab3-b31e-f85ddd82b604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:57.107826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e045606-b296-4ab3-b31e-f85ddd82b604 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:57.108104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e045606-b296-4ab3-b31e-f85ddd82b604 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:57.108375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e045606-b296-4ab3-b31e-f85ddd82b604 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:57.109042 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e045606-b296-4ab3-b31e-f85ddd82b604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:57.109346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e045606-b296-4ab3-b31e-f85ddd82b604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:57.110755 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e045606-b296-4ab3-b31e-f85ddd82b604 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:57.129135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 526/1042] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:57.130619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b2dc0-a16c-412e-8a57-40cd1d3a619e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:57.147285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:57.231492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf00f73-a37a-4bf8-b161-92bffdc1b61b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:57.231827 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf00f73-a37a-4bf8-b161-92bffdc1b61b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:57.232061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf00f73-a37a-4bf8-b161-92bffdc1b61b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:57.233531 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 527/1043] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:02:57.235076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b2dc0-a16c-412e-8a57-40cd1d3a619e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:57.238398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca8c59-10ed-408b-b321-f495ee4610ce None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:57.238682 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca8c59-10ed-408b-b321-f495ee4610ce None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:57.238957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca8c59-10ed-408b-b321-f495ee4610ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:57.246715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:57.257521 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7eca8c59-10ed-408b-b321-f495ee4610ce None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:57.258679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7eca8c59-10ed-408b-b321-f495ee4610ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:57.279684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840817d-0932-41fc-b6ac-5304b99bc856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:57.280741 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840817d-0932-41fc-b6ac-5304b99bc856 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:57.281027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840817d-0932-41fc-b6ac-5304b99bc856 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:57.281304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840817d-0932-41fc-b6ac-5304b99bc856 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:57.290307 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840817d-0932-41fc-b6ac-5304b99bc856 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:57.292135 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840817d-0932-41fc-b6ac-5304b99bc856 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c840817d-0932-41fc-b6ac-5304b99bc856 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:02:57.303730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:57.304648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:57.306205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:02:57.306205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:02:57.306205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:02:57.306205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:02:57.306205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:02:57.306205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:02:57.306205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:02:57.306205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:57.306205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 517/1044] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:02:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:57.310844 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840817d-0932-41fc-b6ac-5304b99bc856 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:57.335163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 528/1045] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:57.389575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:57.420392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840817d-0932-41fc-b6ac-5304b99bc856 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:57.454914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f0694-1e7a-4190-a255-73b8686947e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:57.455562 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f0694-1e7a-4190-a255-73b8686947e7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:57.455840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f0694-1e7a-4190-a255-73b8686947e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:57.456127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f0694-1e7a-4190-a255-73b8686947e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:57.456809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f0694-1e7a-4190-a255-73b8686947e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:57.457114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f0694-1e7a-4190-a255-73b8686947e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:57.458573 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f0694-1e7a-4190-a255-73b8686947e7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:57.476258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 518/1046] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:02:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:57.477219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca8c59-10ed-408b-b321-f495ee4610ce None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:57.577651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca8c59-10ed-408b-b321-f495ee4610ce None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:57.622109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8f36d-598e-44cc-bad9-ee1e0808674e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:57.622725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f36d-598e-44cc-bad9-ee1e0808674e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:57.622999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f36d-598e-44cc-bad9-ee1e0808674e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:57.623290 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f36d-598e-44cc-bad9-ee1e0808674e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:57.624013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8f36d-598e-44cc-bad9-ee1e0808674e None admin] RBAC: Adding query filter params (role.id=bd6892add1ee40ac8720e915e153c34f, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=441575b8b496457b905e9d5f5846e092) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:57.624318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8f36d-598e-44cc-bad9-ee1e0808674e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:57.625736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8f36d-598e-44cc-bad9-ee1e0808674e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:57.627530 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:57.629690 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f0694-1e7a-4190-a255-73b8686947e7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:57.637993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 529/1047] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:57 2022] GET /identity/v3/role_assignments?user.id=441575b8b496457b905e9d5f5846e092&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=bd6892add1ee40ac8720e915e153c34f => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:57.743567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f0694-1e7a-4190-a255-73b8686947e7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:57.777042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:57.783133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0d26d-527e-493b-ae12-d600ac439478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:57.783784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0d26d-527e-493b-ae12-d600ac439478 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:57.784060 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0d26d-527e-493b-ae12-d600ac439478 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:57.784327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0d26d-527e-493b-ae12-d600ac439478 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:57.796459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0d26d-527e-493b-ae12-d600ac439478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-22d0d26d-527e-493b-ae12-d600ac439478 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:02:57.801315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:57.802291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:57.803179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:57.803179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:57.803179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:57.803179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:57.803179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:02:57.803179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:02:57.803179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:02:57.803179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:02:57.803179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:02:57.803179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:02:57.803179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:02:57.803179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:02:57.803179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 519/1048] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:02:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:02:57.809493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8f36d-598e-44cc-bad9-ee1e0808674e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:57.922719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8f36d-598e-44cc-bad9-ee1e0808674e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:57.958347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36f99f-d305-435f-89c2-71c1ab3057f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:57.958967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36f99f-d305-435f-89c2-71c1ab3057f4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:57.959278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36f99f-d305-435f-89c2-71c1ab3057f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:57.959554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36f99f-d305-435f-89c2-71c1ab3057f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:57.960229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36f99f-d305-435f-89c2-71c1ab3057f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:57.960532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36f99f-d305-435f-89c2-71c1ab3057f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:57.961901 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36f99f-d305-435f-89c2-71c1ab3057f4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:57.962524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd5de-3fb7-4316-bcb4-9869374815cf None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:57.962806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd5de-3fb7-4316-bcb4-9869374815cf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:57.963104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd5de-3fb7-4316-bcb4-9869374815cf None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:57.965244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 520/1049] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:57.970371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba7b69-a7c6-4397-81c8-3c76aa9e7d1d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:57.970657 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba7b69-a7c6-4397-81c8-3c76aa9e7d1d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:57.970936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba7b69-a7c6-4397-81c8-3c76aa9e7d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:57.977652 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 530/1050] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:57.979020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36f99f-d305-435f-89c2-71c1ab3057f4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:57.990566 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-48ba7b69-a7c6-4397-81c8-3c76aa9e7d1d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:57.992483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-48ba7b69-a7c6-4397-81c8-3c76aa9e7d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:58.044073 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:58.078686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 521/1051] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:02:58.088548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36f99f-d305-435f-89c2-71c1ab3057f4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:58.125887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec2b80-d103-44c9-a983-d2b03f991a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:58.126582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec2b80-d103-44c9-a983-d2b03f991a22 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:58.126868 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec2b80-d103-44c9-a983-d2b03f991a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:58.127182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec2b80-d103-44c9-a983-d2b03f991a22 None admin] PATH_INFO: `/v3/users/b9a7a7e958d84273b052f5ab3a976d2e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:58.141851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec2b80-d103-44c9-a983-d2b03f991a22 None admin] RBAC: Authorizing `identity:get_user(user_id=b9a7a7e958d84273b052f5ab3a976d2e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:58.143329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec2b80-d103-44c9-a983-d2b03f991a22 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:58.149818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 531/1052] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:02:57 2022] GET /identity/v3/users/b9a7a7e958d84273b052f5ab3a976d2e => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:58.150797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba7b69-a7c6-4397-81c8-3c76aa9e7d1d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:58.252655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba7b69-a7c6-4397-81c8-3c76aa9e7d1d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:58.276980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:58.289881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a95332-94b9-473f-85c7-6ce4d8ad8745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:58.290497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95332-94b9-473f-85c7-6ce4d8ad8745 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:58.291027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95332-94b9-473f-85c7-6ce4d8ad8745 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:58.291027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95332-94b9-473f-85c7-6ce4d8ad8745 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:58.362255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a95332-94b9-473f-85c7-6ce4d8ad8745 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:58.363748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a95332-94b9-473f-85c7-6ce4d8ad8745 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:58.372106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec2b80-d103-44c9-a983-d2b03f991a22 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:58.473251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 522/1053] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:02:58 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:02:58.478931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a95332-94b9-473f-85c7-6ce4d8ad8745 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:58.488760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec2b80-d103-44c9-a983-d2b03f991a22 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:58.506640 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:58.510254 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:58.530559 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ad2f9-de09-4e2c-af06-a683ff48e522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:58.531359 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ad2f9-de09-4e2c-af06-a683ff48e522 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:58.531530 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ad2f9-de09-4e2c-af06-a683ff48e522 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:58.531797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ad2f9-de09-4e2c-af06-a683ff48e522 None admin] PATH_INFO: `/v3/services/image` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:58.532396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ad2f9-de09-4e2c-af06-a683ff48e522 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:58.533974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ad2f9-de09-4e2c-af06-a683ff48e522 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-482ad2f9-de09-4e2c-af06-a683ff48e522 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:02:58.545719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:02:58.546759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:02:58.548302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 532/1054] 10.222.0.17 () {56 vars in 1105 bytes} [Fri Sep 9 13:02:58 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:02:58.553612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ad2f9-de09-4e2c-af06-a683ff48e522 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:58.596195 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a95332-94b9-473f-85c7-6ce4d8ad8745 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:58.635969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86375c06-219c-4ef5-ad21-6948bb65d6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:58.636697 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86375c06-219c-4ef5-ad21-6948bb65d6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:58.636974 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86375c06-219c-4ef5-ad21-6948bb65d6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:58.637244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86375c06-219c-4ef5-ad21-6948bb65d6d8 None admin] PATH_INFO: `/v3/projects/035c9529969d48f3899132d7b3bc15e0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:58.638352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86375c06-219c-4ef5-ad21-6948bb65d6d8 None admin] RBAC: Authorizing `identity:get_project(project_id=035c9529969d48f3899132d7b3bc15e0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:58.640261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86375c06-219c-4ef5-ad21-6948bb65d6d8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:58.642799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 523/1055] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:02:58 2022] GET /identity/v3/projects/035c9529969d48f3899132d7b3bc15e0 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:58.648669 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86375c06-219c-4ef5-ad21-6948bb65d6d8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:58.670767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ad2f9-de09-4e2c-af06-a683ff48e522 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:58.705253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f0a50-d21d-46cb-8a4d-cab7c043fb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:58.705858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f0a50-d21d-46cb-8a4d-cab7c043fb7f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:58.706133 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f0a50-d21d-46cb-8a4d-cab7c043fb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:58.706402 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f0a50-d21d-46cb-8a4d-cab7c043fb7f None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:58.707082 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f0a50-d21d-46cb-8a4d-cab7c043fb7f None admin] RBAC: Adding query filter params (name=image) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:58.707544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f0a50-d21d-46cb-8a4d-cab7c043fb7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:58.708767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f0a50-d21d-46cb-8a4d-cab7c043fb7f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:58.719037 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 533/1056] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:58.719902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f0a50-d21d-46cb-8a4d-cab7c043fb7f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:58.747618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:58.758804 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86375c06-219c-4ef5-ad21-6948bb65d6d8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:58.770865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:58.794096 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553d29e-0a32-4eec-9750-6799670386b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:58.794618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553d29e-0a32-4eec-9750-6799670386b5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:58.794898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553d29e-0a32-4eec-9750-6799670386b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:58.795203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553d29e-0a32-4eec-9750-6799670386b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:58.795926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553d29e-0a32-4eec-9750-6799670386b5 None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=035c9529969d48f3899132d7b3bc15e0, user.id=b9a7a7e958d84273b052f5ab3a976d2e) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:58.796235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553d29e-0a32-4eec-9750-6799670386b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:58.797666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553d29e-0a32-4eec-9750-6799670386b5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:58.815091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 524/1057] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:02:58 2022] GET /identity/v3/role_assignments?user.id=b9a7a7e958d84273b052f5ab3a976d2e&scope.project.id=035c9529969d48f3899132d7b3bc15e0&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:58.845933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f0a50-d21d-46cb-8a4d-cab7c043fb7f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:58.883990 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc67d1c-8653-4f03-9f39-cf6f6342d1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:58.884618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc67d1c-8653-4f03-9f39-cf6f6342d1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:58.884895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc67d1c-8653-4f03-9f39-cf6f6342d1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:58.885163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc67d1c-8653-4f03-9f39-cf6f6342d1a7 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:58.886646 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc67d1c-8653-4f03-9f39-cf6f6342d1a7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:58.887180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc67d1c-8653-4f03-9f39-cf6f6342d1a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:58.888590 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc67d1c-8653-4f03-9f39-cf6f6342d1a7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:58.894221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 534/1058] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:02:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:58.900318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553d29e-0a32-4eec-9750-6799670386b5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:59.012643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553d29e-0a32-4eec-9750-6799670386b5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:59.047836 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135b443-c553-49f1-a091-5bc8db9646bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:59.048444 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135b443-c553-49f1-a091-5bc8db9646bc None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.048719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135b443-c553-49f1-a091-5bc8db9646bc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.048986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135b443-c553-49f1-a091-5bc8db9646bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.049642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135b443-c553-49f1-a091-5bc8db9646bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:59.051000 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135b443-c553-49f1-a091-5bc8db9646bc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:59.079077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 525/1059] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:59.818369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb9823-cb33-4a69-b215-414cb3433c03 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.818883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb9823-cb33-4a69-b215-414cb3433c03 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.819010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb9823-cb33-4a69-b215-414cb3433c03 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.820342 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 535/1060] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:59.826156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e74f6-e647-4412-b779-06fc6b2ad23e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.826580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e74f6-e647-4412-b779-06fc6b2ad23e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.826762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e74f6-e647-4412-b779-06fc6b2ad23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.845135 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c50e74f6-e647-4412-b779-06fc6b2ad23e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:59.846412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c50e74f6-e647-4412-b779-06fc6b2ad23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:59.849269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:59.933386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 526/1061] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:02:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:00.216312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb9823-cb33-4a69-b215-414cb3433c03 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:00.332186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb9823-cb33-4a69-b215-414cb3433c03 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:00.345523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:00.371316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe4671-b62f-4f2e-89cb-6298d8617e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:00.372221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe4671-b62f-4f2e-89cb-6298d8617e68 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.372502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe4671-b62f-4f2e-89cb-6298d8617e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.372772 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe4671-b62f-4f2e-89cb-6298d8617e68 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.383600 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe4671-b62f-4f2e-89cb-6298d8617e68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-5bfe4671-b62f-4f2e-89cb-6298d8617e68 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:03:00.389039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:00.390003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:00.390904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:00.390904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:00.390904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:00.390904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:00.390904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:03:00.390904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:03:00.390904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:03:00.390904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:03:00.390904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:03:00.390904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:03:00.390904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 13:03:00.390904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:03:00.392316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 536/1062] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 9 13:03:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:00.396104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e74f6-e647-4412-b779-06fc6b2ad23e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:00.497473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e74f6-e647-4412-b779-06fc6b2ad23e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:00.532140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4c714-c9e1-4c32-a434-398712f8e36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:00.532750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4c714-c9e1-4c32-a434-398712f8e36f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.533061 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4c714-c9e1-4c32-a434-398712f8e36f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.533288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4c714-c9e1-4c32-a434-398712f8e36f None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.533948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4c714-c9e1-4c32-a434-398712f8e36f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:00.534247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4c714-c9e1-4c32-a434-398712f8e36f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:00.535649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4c714-c9e1-4c32-a434-398712f8e36f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:00.537955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:00.549731 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 527/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 13:03:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:00.551000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe4671-b62f-4f2e-89cb-6298d8617e68 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:00.660826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe4671-b62f-4f2e-89cb-6298d8617e68 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:00.695436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef962c2-6875-428c-befb-ebfd98a1f540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:00.696067 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef962c2-6875-428c-befb-ebfd98a1f540 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.696347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef962c2-6875-428c-befb-ebfd98a1f540 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.696614 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef962c2-6875-428c-befb-ebfd98a1f540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.707670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef962c2-6875-428c-befb-ebfd98a1f540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:00.709115 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef962c2-6875-428c-befb-ebfd98a1f540 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0ef962c2-6875-428c-befb-ebfd98a1f540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:03:00.723023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:00.724025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:03:00.724941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:03:00.724941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:03:00.724941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:03:00.724941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 537/1064] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:00.732018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4c714-c9e1-4c32-a434-398712f8e36f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:00.833168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4c714-c9e1-4c32-a434-398712f8e36f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:00.867035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e2063-cc8a-467e-ae3b-fcbae253a069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:00.867666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e2063-cc8a-467e-ae3b-fcbae253a069 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.867945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e2063-cc8a-467e-ae3b-fcbae253a069 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.868215 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e2063-cc8a-467e-ae3b-fcbae253a069 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.868850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e2063-cc8a-467e-ae3b-fcbae253a069 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:00.869151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e2063-cc8a-467e-ae3b-fcbae253a069 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:00.871228 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e2063-cc8a-467e-ae3b-fcbae253a069 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:00.888078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 528/1065] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:00.889458 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef962c2-6875-428c-befb-ebfd98a1f540 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:00.918083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:01.000657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef962c2-6875-428c-befb-ebfd98a1f540 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:01.015608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b151401-d285-419d-8135-79ea097010fc None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.015891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b151401-d285-419d-8135-79ea097010fc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.016161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b151401-d285-419d-8135-79ea097010fc None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.017501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 529/1066] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:03:01.022323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72504b6-8a81-4815-ba39-13641a0399a6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.022600 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72504b6-8a81-4815-ba39-13641a0399a6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.022870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72504b6-8a81-4815-ba39-13641a0399a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.038691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87b272-f6ab-4625-8e03-94dde027feb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:01.039566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87b272-f6ab-4625-8e03-94dde027feb7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.039995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87b272-f6ab-4625-8e03-94dde027feb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.040473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87b272-f6ab-4625-8e03-94dde027feb7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.040989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d72504b6-8a81-4815-ba39-13641a0399a6 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:01.042989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d72504b6-8a81-4815-ba39-13641a0399a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:01.050971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87b272-f6ab-4625-8e03-94dde027feb7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:01.052903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87b272-f6ab-4625-8e03-94dde027feb7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-af87b272-f6ab-4625-8e03-94dde027feb7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:03:01.069154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:01.070101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:01.071501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:01.071501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:01.071501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:03:01.071501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:03:01.071501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:03:01.071501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:03:01.071501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:03:01.071501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:03:01.071501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:03:01.071501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:03:01.072378 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 538/1067] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 13:03:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:01.077211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87b272-f6ab-4625-8e03-94dde027feb7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:01.105766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:01.121282 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 530/1068] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:03:01.170957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:01.192183 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87b272-f6ab-4625-8e03-94dde027feb7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:01.226871 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6c004-5607-479e-a77f-f2cacade5c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:01.227506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6c004-5607-479e-a77f-f2cacade5c14 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.227804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6c004-5607-479e-a77f-f2cacade5c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.228069 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6c004-5607-479e-a77f-f2cacade5c14 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.228792 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6c004-5607-479e-a77f-f2cacade5c14 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:01.229145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6c004-5607-479e-a77f-f2cacade5c14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:01.230777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6c004-5607-479e-a77f-f2cacade5c14 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:01.254365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 539/1069] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 13:03:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:03:01.255755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72504b6-8a81-4815-ba39-13641a0399a6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:01.268703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ac9d-c551-4d06-a636-e14fab877897 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.269081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ac9d-c551-4d06-a636-e14fab877897 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.269452 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ac9d-c551-4d06-a636-e14fab877897 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.270996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 540/1070] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:03:01.276493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07346ca5-f6c2-4166-b580-3bffbbc1a7bd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.276929 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07346ca5-f6c2-4166-b580-3bffbbc1a7bd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.277345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07346ca5-f6c2-4166-b580-3bffbbc1a7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.297339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-07346ca5-f6c2-4166-b580-3bffbbc1a7bd None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:01.298870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-07346ca5-f6c2-4166-b580-3bffbbc1a7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:01.364382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72504b6-8a81-4815-ba39-13641a0399a6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:01.370943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 541/1071] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:03:01.394474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07346ca5-f6c2-4166-b580-3bffbbc1a7bd None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:01.401824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22035f8e-4d46-42d8-9d53-3ff3884023e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:01.402463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22035f8e-4d46-42d8-9d53-3ff3884023e9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.402742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22035f8e-4d46-42d8-9d53-3ff3884023e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.403013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22035f8e-4d46-42d8-9d53-3ff3884023e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.414750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22035f8e-4d46-42d8-9d53-3ff3884023e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:01.416440 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22035f8e-4d46-42d8-9d53-3ff3884023e9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-22035f8e-4d46-42d8-9d53-3ff3884023e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:03:01.433057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:01.434085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:03:01.435344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:03:01.435344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:03:01.435344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:03:01.435344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 531/1072] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:01.439777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22035f8e-4d46-42d8-9d53-3ff3884023e9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:01.475436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:01.481599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:01.509593 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07346ca5-f6c2-4166-b580-3bffbbc1a7bd None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:01.543431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22035f8e-4d46-42d8-9d53-3ff3884023e9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:01.545320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3de28e-5694-4b3a-a48b-2a5aea9364e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:01.546322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de28e-5694-4b3a-a48b-2a5aea9364e3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.546322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de28e-5694-4b3a-a48b-2a5aea9364e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.546523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de28e-5694-4b3a-a48b-2a5aea9364e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.556977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3de28e-5694-4b3a-a48b-2a5aea9364e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7b3de28e-5694-4b3a-a48b-2a5aea9364e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:03:01.562576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:01.563546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:01.568113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:01.568113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:01.568113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:01.568113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:01.568113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:03:01.568113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:03:01.568113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:03:01.568113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:03:01.568113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:03:01.568113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:03:01.568113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:01.568113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:03:01.568113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 542/1073] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:03:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:03:01.570016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3de28e-5694-4b3a-a48b-2a5aea9364e3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:01.586295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd27ed-372b-44b1-aeb3-3a83c9825dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:01.586965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd27ed-372b-44b1-aeb3-3a83c9825dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.587272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd27ed-372b-44b1-aeb3-3a83c9825dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.587544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd27ed-372b-44b1-aeb3-3a83c9825dc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.588255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd27ed-372b-44b1-aeb3-3a83c9825dc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:01.588556 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd27ed-372b-44b1-aeb3-3a83c9825dc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:01.590114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd27ed-372b-44b1-aeb3-3a83c9825dc8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:01.605788 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 532/1074] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:03:01.611007 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd27ed-372b-44b1-aeb3-3a83c9825dc8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:01.683519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3de28e-5694-4b3a-a48b-2a5aea9364e3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:01.707785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:01.710921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd27ed-372b-44b1-aeb3-3a83c9825dc8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:01.711365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:01.717762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c34ca6-4d30-46a0-99b1-15ef35c99543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:01.718372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c34ca6-4d30-46a0-99b1-15ef35c99543 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.718655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c34ca6-4d30-46a0-99b1-15ef35c99543 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.718927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c34ca6-4d30-46a0-99b1-15ef35c99543 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.719623 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c34ca6-4d30-46a0-99b1-15ef35c99543 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:01.719935 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c34ca6-4d30-46a0-99b1-15ef35c99543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:01.721304 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c34ca6-4d30-46a0-99b1-15ef35c99543 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:01.729902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 543/1075] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:03:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:03:01.730695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c34ca6-4d30-46a0-99b1-15ef35c99543 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:01.751850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b221c6a-f006-4b47-bcba-efb7af0aba87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:01.752488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b221c6a-f006-4b47-bcba-efb7af0aba87 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.752766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b221c6a-f006-4b47-bcba-efb7af0aba87 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.753038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b221c6a-f006-4b47-bcba-efb7af0aba87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.759575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b221c6a-f006-4b47-bcba-efb7af0aba87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:01.761679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b221c6a-f006-4b47-bcba-efb7af0aba87 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4b221c6a-f006-4b47-bcba-efb7af0aba87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:03:01.773035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:01.774034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:01.774973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:03:01.774973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:03:01.774973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:03:01.774973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:03:01.774973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:03:01.774973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:03:01.774973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:03:01.774973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:03:01.774973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 533/1076] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:03:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:03:01.775509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b221c6a-f006-4b47-bcba-efb7af0aba87 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:01.833532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c34ca6-4d30-46a0-99b1-15ef35c99543 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:01.867332 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43122cdc-8933-4130-a89e-94a66bcc8263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:01.867858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43122cdc-8933-4130-a89e-94a66bcc8263 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.868137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43122cdc-8933-4130-a89e-94a66bcc8263 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.868405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43122cdc-8933-4130-a89e-94a66bcc8263 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.874950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b221c6a-f006-4b47-bcba-efb7af0aba87 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:01.909325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa87277-848f-4147-bb6b-4fa866d0ec36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:01.909955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa87277-848f-4147-bb6b-4fa866d0ec36 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.910234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa87277-848f-4147-bb6b-4fa866d0ec36 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.910504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa87277-848f-4147-bb6b-4fa866d0ec36 None admin] PATH_INFO: `/v3/users/b9a7a7e958d84273b052f5ab3a976d2e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.911654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa87277-848f-4147-bb6b-4fa866d0ec36 None admin] RBAC: Authorizing `identity:get_user(user_id=b9a7a7e958d84273b052f5ab3a976d2e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:01.913083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa87277-848f-4147-bb6b-4fa866d0ec36 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:01.914681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 534/1077] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:01 2022] GET /identity/v3/users/b9a7a7e958d84273b052f5ab3a976d2e => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:01.915458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa87277-848f-4147-bb6b-4fa866d0ec36 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:01.930143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:01.940844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43122cdc-8933-4130-a89e-94a66bcc8263 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:01.942293 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43122cdc-8933-4130-a89e-94a66bcc8263 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:01.946027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:02.018585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa87277-848f-4147-bb6b-4fa866d0ec36 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:02.041085 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 544/1078] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:03:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:03:02.041934 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43122cdc-8933-4130-a89e-94a66bcc8263 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:02.054668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a102325-3d9a-438d-a604-59a4700b292a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:02.055318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a102325-3d9a-438d-a604-59a4700b292a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:02.055597 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a102325-3d9a-438d-a604-59a4700b292a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:02.055868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a102325-3d9a-438d-a604-59a4700b292a None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:02.056560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a102325-3d9a-438d-a604-59a4700b292a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:02.057369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a102325-3d9a-438d-a604-59a4700b292a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:02.059078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a102325-3d9a-438d-a604-59a4700b292a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:02.071733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 535/1079] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:03:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:02.072654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a102325-3d9a-438d-a604-59a4700b292a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:02.135966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:02.145700 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43122cdc-8933-4130-a89e-94a66bcc8263 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:02.159971 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:02.172812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a102325-3d9a-438d-a604-59a4700b292a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:02.180834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7369ae-2029-49e4-9a8b-9bea9659f889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:02.181463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7369ae-2029-49e4-9a8b-9bea9659f889 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:02.181745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7369ae-2029-49e4-9a8b-9bea9659f889 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:02.182152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7369ae-2029-49e4-9a8b-9bea9659f889 None admin] PATH_INFO: `/v3/projects/035c9529969d48f3899132d7b3bc15e0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:02.183558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7369ae-2029-49e4-9a8b-9bea9659f889 None admin] RBAC: Authorizing `identity:get_project(project_id=035c9529969d48f3899132d7b3bc15e0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:02.185000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7369ae-2029-49e4-9a8b-9bea9659f889 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:02.186650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 545/1080] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:03:02 2022] GET /identity/v3/projects/035c9529969d48f3899132d7b3bc15e0 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:02.187416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7369ae-2029-49e4-9a8b-9bea9659f889 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:02.212021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdae8fe-0592-47b8-866a-c0f057e66cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:02.212648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdae8fe-0592-47b8-866a-c0f057e66cfd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:02.212927 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdae8fe-0592-47b8-866a-c0f057e66cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:02.213255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdae8fe-0592-47b8-866a-c0f057e66cfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:02.223735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdae8fe-0592-47b8-866a-c0f057e66cfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:02.225179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdae8fe-0592-47b8-866a-c0f057e66cfd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1cdae8fe-0592-47b8-866a-c0f057e66cfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:03:02.239681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:02.240730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:03:02.241662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:03:02.241662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:03:02.241662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:03:02.241662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 536/1081] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:02.241662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdae8fe-0592-47b8-866a-c0f057e66cfd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:02.294454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7369ae-2029-49e4-9a8b-9bea9659f889 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:02.328543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a63fb9-6610-4a8b-a8a1-9eaf502faa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:02.329330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a63fb9-6610-4a8b-a8a1-9eaf502faa99 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:02.329615 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a63fb9-6610-4a8b-a8a1-9eaf502faa99 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:02.329888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a63fb9-6610-4a8b-a8a1-9eaf502faa99 None admin] PATH_INFO: `/v3/projects/2b842ac074b44e148c15882a7964aa13/users/441575b8b496457b905e9d5f5846e092/roles/bd6892add1ee40ac8720e915e153c34f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:02.332110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a63fb9-6610-4a8b-a8a1-9eaf502faa99 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b842ac074b44e148c15882a7964aa13, user_id=441575b8b496457b905e9d5f5846e092, role_id=bd6892add1ee40ac8720e915e153c34f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:02.333575 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a63fb9-6610-4a8b-a8a1-9eaf502faa99 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:02.343336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdae8fe-0592-47b8-866a-c0f057e66cfd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:02.346889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 546/1082] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:03:02 2022] PUT /identity/v3/projects/2b842ac074b44e148c15882a7964aa13/users/441575b8b496457b905e9d5f5846e092/roles/bd6892add1ee40ac8720e915e153c34f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:02.347712 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a63fb9-6610-4a8b-a8a1-9eaf502faa99 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:02.372921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:02.384622 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0c1f8-bd43-4591-a03e-7928339a93c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:02.385963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0c1f8-bd43-4591-a03e-7928339a93c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:02.386311 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0c1f8-bd43-4591-a03e-7928339a93c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:02.386514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0c1f8-bd43-4591-a03e-7928339a93c8 None admin] PATH_INFO: `/v3/projects/035c9529969d48f3899132d7b3bc15e0/users/b9a7a7e958d84273b052f5ab3a976d2e/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:02.388837 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0c1f8-bd43-4591-a03e-7928339a93c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=035c9529969d48f3899132d7b3bc15e0, user_id=b9a7a7e958d84273b052f5ab3a976d2e, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:02.390413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0c1f8-bd43-4591-a03e-7928339a93c8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:02.404068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:02.423758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 537/1083] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:03:02 2022] PUT /identity/v3/projects/035c9529969d48f3899132d7b3bc15e0/users/b9a7a7e958d84273b052f5ab3a976d2e/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:02.484240 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a63fb9-6610-4a8b-a8a1-9eaf502faa99 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:02.525399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec946d8e-167c-4867-8011-c3b11200ad57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:02.526253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec946d8e-167c-4867-8011-c3b11200ad57 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:02.526540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec946d8e-167c-4867-8011-c3b11200ad57 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:02.526828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec946d8e-167c-4867-8011-c3b11200ad57 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:02.527528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec946d8e-167c-4867-8011-c3b11200ad57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:02.528024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec946d8e-167c-4867-8011-c3b11200ad57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:02.529558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec946d8e-167c-4867-8011-c3b11200ad57 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:02.545264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 547/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:04.599746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c152c-f406-4e2d-991e-4599806d2568 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:04.600294 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c152c-f406-4e2d-991e-4599806d2568 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:04.600397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340d743-d1a5-4cf8-9644-91374a2d767c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:04.600768 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c152c-f406-4e2d-991e-4599806d2568 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:04.600867 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340d743-d1a5-4cf8-9644-91374a2d767c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:04.600867 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340d743-d1a5-4cf8-9644-91374a2d767c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:04.601617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 538/1085] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:04.602014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 548/1086] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:03:04.607427 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e8758-12bd-4b82-8feb-85a6d45a827e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:04.607713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e8758-12bd-4b82-8feb-85a6d45a827e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:04.607826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e594c-8966-4fc5-ae1f-3490847b1dea None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:04.607993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e8758-12bd-4b82-8feb-85a6d45a827e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:04.608108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e594c-8966-4fc5-ae1f-3490847b1dea None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:04.608368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e594c-8966-4fc5-ae1f-3490847b1dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:04.626810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ad2e8758-12bd-4b82-8feb-85a6d45a827e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:04.627009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f09e594c-8966-4fc5-ae1f-3490847b1dea None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:04.628056 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ad2e8758-12bd-4b82-8feb-85a6d45a827e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:04.628165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f09e594c-8966-4fc5-ae1f-3490847b1dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:04.694867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 539/1087] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:04.695430 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 549/1088] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:03:04.964983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e8758-12bd-4b82-8feb-85a6d45a827e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:04.971700 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e594c-8966-4fc5-ae1f-3490847b1dea None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:04.984038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:05.065402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e8758-12bd-4b82-8feb-85a6d45a827e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:05.072971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e594c-8966-4fc5-ae1f-3490847b1dea None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:05.093557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:05.099542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a746a3-1171-47a3-a1e8-094f512c2913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:05.100173 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a746a3-1171-47a3-a1e8-094f512c2913 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:05.100446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a746a3-1171-47a3-a1e8-094f512c2913 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:05.100714 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a746a3-1171-47a3-a1e8-094f512c2913 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:05.106503 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364ae0d-0a95-42c5-8f9f-d4684f677ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:05.107162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364ae0d-0a95-42c5-8f9f-d4684f677ecf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:05.107446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364ae0d-0a95-42c5-8f9f-d4684f677ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:05.107722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364ae0d-0a95-42c5-8f9f-d4684f677ecf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:05.111109 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a746a3-1171-47a3-a1e8-094f512c2913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-68a746a3-1171-47a3-a1e8-094f512c2913 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:03:05.115799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:05.117015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:05.118280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:05.118280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:05.118280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:05.118280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:05.118280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:03:05.118280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:03:05.118280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:03:05.118280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:03:05.118280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:03:05.118280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:03:05.118280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 13:03:05.118280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:03:05.118280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 540/1089] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 9 13:03:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:03:05.119805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364ae0d-0a95-42c5-8f9f-d4684f677ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:03:05.123388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a746a3-1171-47a3-a1e8-094f512c2913 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-3364ae0d-0a95-42c5-8f9f-d4684f677ecf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:03:05.123599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:05.124875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:05.125917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:05.125917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:05.125917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:05.125917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:05.125917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:03:05.125917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:03:05.125917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:03:05.125917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:03:05.125917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:03:05.125917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:03:05.125917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:05.125917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:03:05.125917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 550/1090] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:03:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:05.130417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364ae0d-0a95-42c5-8f9f-d4684f677ecf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:05.201552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:05.221841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a746a3-1171-47a3-a1e8-094f512c2913 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:05.233337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364ae0d-0a95-42c5-8f9f-d4684f677ecf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:05.257272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74208b2-17dd-4dc7-b7a2-f25bcf6b2c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:05.257882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74208b2-17dd-4dc7-b7a2-f25bcf6b2c9b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:05.258159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74208b2-17dd-4dc7-b7a2-f25bcf6b2c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:05.258431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74208b2-17dd-4dc7-b7a2-f25bcf6b2c9b None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:05.259117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74208b2-17dd-4dc7-b7a2-f25bcf6b2c9b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:05.259428 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74208b2-17dd-4dc7-b7a2-f25bcf6b2c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:05.260981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74208b2-17dd-4dc7-b7a2-f25bcf6b2c9b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:05.268245 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efea7cc-14cc-4255-aeec-d125649cee73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:05.268779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 541/1091] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 13:03:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:03:05.268923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efea7cc-14cc-4255-aeec-d125649cee73 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:05.269108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efea7cc-14cc-4255-aeec-d125649cee73 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:05.269380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efea7cc-14cc-4255-aeec-d125649cee73 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:05.270038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efea7cc-14cc-4255-aeec-d125649cee73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:05.270341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efea7cc-14cc-4255-aeec-d125649cee73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:05.271721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efea7cc-14cc-4255-aeec-d125649cee73 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:05.274890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74208b2-17dd-4dc7-b7a2-f25bcf6b2c9b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:05.280225 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 551/1092] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:03:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:03:05.285881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efea7cc-14cc-4255-aeec-d125649cee73 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:05.326930 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:05.375494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74208b2-17dd-4dc7-b7a2-f25bcf6b2c9b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:05.385192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efea7cc-14cc-4255-aeec-d125649cee73 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:05.408618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b6331-2aae-4205-908e-6168974bd2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:05.409246 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b6331-2aae-4205-908e-6168974bd2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:05.409525 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b6331-2aae-4205-908e-6168974bd2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:05.409797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b6331-2aae-4205-908e-6168974bd2c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:05.413642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:05.419665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55515f9a-8500-4d2e-ac61-018f6f7e7ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:05.420294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55515f9a-8500-4d2e-ac61-018f6f7e7ade None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:05.420577 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55515f9a-8500-4d2e-ac61-018f6f7e7ade None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:05.420850 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55515f9a-8500-4d2e-ac61-018f6f7e7ade None admin] PATH_INFO: `/v3/users/b9a7a7e958d84273b052f5ab3a976d2e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:05.420947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b6331-2aae-4205-908e-6168974bd2c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:05.422092 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55515f9a-8500-4d2e-ac61-018f6f7e7ade None admin] RBAC: Authorizing `identity:get_user(user_id=b9a7a7e958d84273b052f5ab3a976d2e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:05.422326 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b6331-2aae-4205-908e-6168974bd2c2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:05.423544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55515f9a-8500-4d2e-ac61-018f6f7e7ade None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:05.425155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 552/1093] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:05 2022] GET /identity/v3/users/b9a7a7e958d84273b052f5ab3a976d2e => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:03:05.430728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55515f9a-8500-4d2e-ac61-018f6f7e7ade None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-396b6331-2aae-4205-908e-6168974bd2c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:03:05.437560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:05.438834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:03:05.439936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:03:05.439936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:03:05.439936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:03:05.443214 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 542/1094] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:05.444376 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b6331-2aae-4205-908e-6168974bd2c2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:05.529674 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55515f9a-8500-4d2e-ac61-018f6f7e7ade None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:05.542492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b6331-2aae-4205-908e-6168974bd2c2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:05.545035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:05.564424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2fd0d-088c-4f6c-b90a-fec376026bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:05.565035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2fd0d-088c-4f6c-b90a-fec376026bec None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:05.565315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2fd0d-088c-4f6c-b90a-fec376026bec None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:05.565588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2fd0d-088c-4f6c-b90a-fec376026bec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:05.578310 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a91aeb-5cf9-4daa-91c4-0d160192ceda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:05.578919 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a91aeb-5cf9-4daa-91c4-0d160192ceda None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:05.579229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a91aeb-5cf9-4daa-91c4-0d160192ceda None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:05.579500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a91aeb-5cf9-4daa-91c4-0d160192ceda None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:05.580139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a91aeb-5cf9-4daa-91c4-0d160192ceda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:05.580443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a91aeb-5cf9-4daa-91c4-0d160192ceda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:05.581811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a91aeb-5cf9-4daa-91c4-0d160192ceda None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:05.594276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 543/1095] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:03:05.599835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a91aeb-5cf9-4daa-91c4-0d160192ceda None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:05.637895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:05.640356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2fd0d-088c-4f6c-b90a-fec376026bec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:05.641879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2fd0d-088c-4f6c-b90a-fec376026bec None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:05.699737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a91aeb-5cf9-4daa-91c4-0d160192ceda None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:05.733144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f797e9-a4f0-4e73-81ab-8e42858c4b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:05.733774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f797e9-a4f0-4e73-81ab-8e42858c4b7e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:05.734053 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f797e9-a4f0-4e73-81ab-8e42858c4b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:05.734326 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f797e9-a4f0-4e73-81ab-8e42858c4b7e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:05.738948 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 553/1096] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:03:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 13:03:05.741262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f797e9-a4f0-4e73-81ab-8e42858c4b7e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:05.742709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f797e9-a4f0-4e73-81ab-8e42858c4b7e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:05.745349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2fd0d-088c-4f6c-b90a-fec376026bec None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d0f797e9-a4f0-4e73-81ab-8e42858c4b7e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:03:05.755092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:05.756175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:05.757594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:05.757594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:05.757594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:03:05.757594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:03:05.757594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:03:05.757594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:03:05.757594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:03:05.757594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:03:05.757594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 13:03:05.757594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:03:05.757594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 544/1097] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 9 13:03:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:03:05.762363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f797e9-a4f0-4e73-81ab-8e42858c4b7e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:05.786621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:05.844893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:05.847945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2fd0d-088c-4f6c-b90a-fec376026bec None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:05.865417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f797e9-a4f0-4e73-81ab-8e42858c4b7e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:05.882286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a5e6c-2089-46ea-bf35-ebc21abfd093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:05.882913 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a5e6c-2089-46ea-bf35-ebc21abfd093 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:05.883226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a5e6c-2089-46ea-bf35-ebc21abfd093 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:05.883496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a5e6c-2089-46ea-bf35-ebc21abfd093 None admin] PATH_INFO: `/v3/projects/035c9529969d48f3899132d7b3bc15e0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:05.884711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a5e6c-2089-46ea-bf35-ebc21abfd093 None admin] RBAC: Authorizing `identity:get_project(project_id=035c9529969d48f3899132d7b3bc15e0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:05.886136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a5e6c-2089-46ea-bf35-ebc21abfd093 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:05.887836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 554/1098] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:03:05 2022] GET /identity/v3/projects/035c9529969d48f3899132d7b3bc15e0 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:03:05.893875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a5e6c-2089-46ea-bf35-ebc21abfd093 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:05.899778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e3b37-7882-450b-8516-60f7fa42dc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:05.901048 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e3b37-7882-450b-8516-60f7fa42dc32 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:05.901329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e3b37-7882-450b-8516-60f7fa42dc32 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:05.901603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e3b37-7882-450b-8516-60f7fa42dc32 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:05.902784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e3b37-7882-450b-8516-60f7fa42dc32 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:05.903142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e3b37-7882-450b-8516-60f7fa42dc32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:05.904939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e3b37-7882-450b-8516-60f7fa42dc32 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:05.923552 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 545/1099] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 9 13:03:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:03:05.936106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e3b37-7882-450b-8516-60f7fa42dc32 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:05.997910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a5e6c-2089-46ea-bf35-ebc21abfd093 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:06.015866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:06.032801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d5c68-25f8-48d4-b42b-d67415a5af0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:06.033410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d5c68-25f8-48d4-b42b-d67415a5af0a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:06.033689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d5c68-25f8-48d4-b42b-d67415a5af0a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:06.033960 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d5c68-25f8-48d4-b42b-d67415a5af0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:06.034683 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d5c68-25f8-48d4-b42b-d67415a5af0a None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=035c9529969d48f3899132d7b3bc15e0, user.id=b9a7a7e958d84273b052f5ab3a976d2e) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:06.034795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e3b37-7882-450b-8516-60f7fa42dc32 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:06.034992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d5c68-25f8-48d4-b42b-d67415a5af0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:06.036452 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d5c68-25f8-48d4-b42b-d67415a5af0a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:06.042291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 555/1100] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:03:05 2022] GET /identity/v3/role_assignments?user.id=b9a7a7e958d84273b052f5ab3a976d2e&scope.project.id=035c9529969d48f3899132d7b3bc15e0&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:03:06.070420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d5bb1-c0d0-4eea-a4e2-574fd3993c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:06.071083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d5bb1-c0d0-4eea-a4e2-574fd3993c54 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:06.071369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d5bb1-c0d0-4eea-a4e2-574fd3993c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:06.071640 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d5bb1-c0d0-4eea-a4e2-574fd3993c54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:06.145567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d5bb1-c0d0-4eea-a4e2-574fd3993c54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:06.147040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d5bb1-c0d0-4eea-a4e2-574fd3993c54 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:06.232682 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:06.251779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 546/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:03:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 13:03:06.257827 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d5c68-25f8-48d4-b42b-d67415a5af0a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:06.290638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:06.359550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d5c68-25f8-48d4-b42b-d67415a5af0a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:06.394421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be7580-4a5e-4f45-b01a-a45bca2d911a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:06.395070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be7580-4a5e-4f45-b01a-a45bca2d911a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:06.395358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be7580-4a5e-4f45-b01a-a45bca2d911a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:06.395629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be7580-4a5e-4f45-b01a-a45bca2d911a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:06.406148 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be7580-4a5e-4f45-b01a-a45bca2d911a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:06.407615 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be7580-4a5e-4f45-b01a-a45bca2d911a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-26be7580-4a5e-4f45-b01a-a45bca2d911a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:03:06.421357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:06.422532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:03:06.423469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:03:06.423469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:03:06.423469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:03:06.423469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 556/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:03:06.428163 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d5bb1-c0d0-4eea-a4e2-574fd3993c54 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:06.525887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d5bb1-c0d0-4eea-a4e2-574fd3993c54 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:06.560170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdb995-7555-43fc-87b5-7dfd42d959e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:06.560777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdb995-7555-43fc-87b5-7dfd42d959e3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:06.561054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdb995-7555-43fc-87b5-7dfd42d959e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:06.561325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdb995-7555-43fc-87b5-7dfd42d959e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:06.561957 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdb995-7555-43fc-87b5-7dfd42d959e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:06.562258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdb995-7555-43fc-87b5-7dfd42d959e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:06.563676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdb995-7555-43fc-87b5-7dfd42d959e3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:06.575819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 547/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:06.581656 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be7580-4a5e-4f45-b01a-a45bca2d911a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:06.671319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:06.687693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be7580-4a5e-4f45-b01a-a45bca2d911a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:06.720488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97010a-b9b4-4e9b-a22c-cd4afd69ad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:06.721108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97010a-b9b4-4e9b-a22c-cd4afd69ad6c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:06.721383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97010a-b9b4-4e9b-a22c-cd4afd69ad6c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:06.721658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97010a-b9b4-4e9b-a22c-cd4afd69ad6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:06.728053 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97010a-b9b4-4e9b-a22c-cd4afd69ad6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:06.730329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97010a-b9b4-4e9b-a22c-cd4afd69ad6c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-dd97010a-b9b4-4e9b-a22c-cd4afd69ad6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:03:06.741134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:06.742113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:06.743131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:03:06.743131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:03:06.743131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:03:06.743131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:03:06.743131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:03:06.743131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:03:06.743131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:03:06.743131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:03:06.743131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 557/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:03:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:06.748765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdb995-7555-43fc-87b5-7dfd42d959e3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:06.792399 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:06.850323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdb995-7555-43fc-87b5-7dfd42d959e3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:06.885315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7493657-0a0b-43f3-bb2a-c5f0804f8744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:06.885858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7493657-0a0b-43f3-bb2a-c5f0804f8744 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:06.886135 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7493657-0a0b-43f3-bb2a-c5f0804f8744 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:06.886408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7493657-0a0b-43f3-bb2a-c5f0804f8744 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:06.887115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7493657-0a0b-43f3-bb2a-c5f0804f8744 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:06.887424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7493657-0a0b-43f3-bb2a-c5f0804f8744 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:06.888846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7493657-0a0b-43f3-bb2a-c5f0804f8744 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:06.902610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 548/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:03:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:06.910293 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97010a-b9b4-4e9b-a22c-cd4afd69ad6c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:07.017980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97010a-b9b4-4e9b-a22c-cd4afd69ad6c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:07.052388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c650e-78b6-492b-b923-8639bcd59d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:07.052987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c650e-78b6-492b-b923-8639bcd59d1a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:07.053263 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c650e-78b6-492b-b923-8639bcd59d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:07.053533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c650e-78b6-492b-b923-8639bcd59d1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:07.054248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c650e-78b6-492b-b923-8639bcd59d1a None admin] RBAC: Adding query filter params (role.id=bd6892add1ee40ac8720e915e153c34f, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=441575b8b496457b905e9d5f5846e092) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:07.054554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c650e-78b6-492b-b923-8639bcd59d1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:07.055998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c650e-78b6-492b-b923-8639bcd59d1a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:07.062662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 558/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:03:06 2022] GET /identity/v3/role_assignments?user.id=441575b8b496457b905e9d5f5846e092&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=bd6892add1ee40ac8720e915e153c34f => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:08.239490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629685e-c0fb-4550-a70e-d8db31db8c9b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:08.240048 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629685e-c0fb-4550-a70e-d8db31db8c9b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:08.240048 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629685e-c0fb-4550-a70e-d8db31db8c9b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:08.241361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 549/1107] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:08.247512 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc08ea-f953-47ef-b0d9-d4980a1d9d93 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:08.247943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc08ea-f953-47ef-b0d9-d4980a1d9d93 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:08.248070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc08ea-f953-47ef-b0d9-d4980a1d9d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:08.266134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b2bc08ea-f953-47ef-b0d9-d4980a1d9d93 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:08.267349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b2bc08ea-f953-47ef-b0d9-d4980a1d9d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:08.269277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:08.334695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 559/1108] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:08.613594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629685e-c0fb-4550-a70e-d8db31db8c9b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:08.713087 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629685e-c0fb-4550-a70e-d8db31db8c9b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:08.714779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:08.748340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b00ab1-d314-4e24-88b6-542928412511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:08.748954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b00ab1-d314-4e24-88b6-542928412511 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:08.749227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b00ab1-d314-4e24-88b6-542928412511 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:08.749494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b00ab1-d314-4e24-88b6-542928412511 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:08.750574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b00ab1-d314-4e24-88b6-542928412511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:08.752023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b00ab1-d314-4e24-88b6-542928412511 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:08.753561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 550/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:03:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:08.759307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc08ea-f953-47ef-b0d9-d4980a1d9d93 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:08.855658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc08ea-f953-47ef-b0d9-d4980a1d9d93 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:08.887607 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7e5dc-d6e1-4a7d-a06e-51afd7db9f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:08.888149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7e5dc-d6e1-4a7d-a06e-51afd7db9f12 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:08.888433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7e5dc-d6e1-4a7d-a06e-51afd7db9f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:08.888701 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7e5dc-d6e1-4a7d-a06e-51afd7db9f12 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:08.889341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7e5dc-d6e1-4a7d-a06e-51afd7db9f12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:08.890753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7e5dc-d6e1-4a7d-a06e-51afd7db9f12 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:08.901396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:08.912288 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 560/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:03:08 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:11.091346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b074a-f1d2-4bf2-b4b2-8c69193c36bd None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:11.091896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b074a-f1d2-4bf2-b4b2-8c69193c36bd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:11.091896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b074a-f1d2-4bf2-b4b2-8c69193c36bd None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:11.093255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 551/1111] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:11.098579 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5062c-4e1d-475e-80f7-88d182f69f71 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:11.098966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5062c-4e1d-475e-80f7-88d182f69f71 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:11.099169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5062c-4e1d-475e-80f7-88d182f69f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:11.117756 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5ea5062c-4e1d-475e-80f7-88d182f69f71 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:11.119540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5ea5062c-4e1d-475e-80f7-88d182f69f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:11.184964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 561/1112] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:11.454091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b074a-f1d2-4bf2-b4b2-8c69193c36bd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:11.553389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b074a-f1d2-4bf2-b4b2-8c69193c36bd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:11.586766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6851e-ca5a-4ad5-ac4e-7c7aeeb5d494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:11.588167 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6851e-ca5a-4ad5-ac4e-7c7aeeb5d494 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:11.588442 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6851e-ca5a-4ad5-ac4e-7c7aeeb5d494 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:11.588709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6851e-ca5a-4ad5-ac4e-7c7aeeb5d494 None admin] PATH_INFO: `/v3/roles/0238ab71fb17471cb87e84097272c752` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:11.590078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6851e-ca5a-4ad5-ac4e-7c7aeeb5d494 None admin] RBAC: Authorizing `identity:get_role(role_id=0238ab71fb17471cb87e84097272c752)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:11.591488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6851e-ca5a-4ad5-ac4e-7c7aeeb5d494 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:11.592815 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 552/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:11 2022] GET /identity/v3/roles/0238ab71fb17471cb87e84097272c752 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:11.598515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5062c-4e1d-475e-80f7-88d182f69f71 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:11.694326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5062c-4e1d-475e-80f7-88d182f69f71 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:11.717163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:11.728349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780ed4e-4584-4ff8-82d4-4e9462942bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:11.728973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780ed4e-4584-4ff8-82d4-4e9462942bdf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:11.729246 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780ed4e-4584-4ff8-82d4-4e9462942bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:11.729512 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780ed4e-4584-4ff8-82d4-4e9462942bdf None admin] PATH_INFO: `/v3/users/a44ff153d124495399c20f127f5c2d8c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:11.742000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780ed4e-4584-4ff8-82d4-4e9462942bdf None admin] RBAC: Authorizing `identity:get_user(user_id=a44ff153d124495399c20f127f5c2d8c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:11.744133 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780ed4e-4584-4ff8-82d4-4e9462942bdf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:11.745783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 562/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:11 2022] GET /identity/v3/users/a44ff153d124495399c20f127f5c2d8c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:11.751678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6851e-ca5a-4ad5-ac4e-7c7aeeb5d494 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:11.790979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:11.850260 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6851e-ca5a-4ad5-ac4e-7c7aeeb5d494 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:11.883429 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a7c52-0928-4108-8073-a6086639501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:11.884574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a7c52-0928-4108-8073-a6086639501f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:11.884850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a7c52-0928-4108-8073-a6086639501f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:11.885122 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a7c52-0928-4108-8073-a6086639501f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:11.949606 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a7c52-0928-4108-8073-a6086639501f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:11.951074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a7c52-0928-4108-8073-a6086639501f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:11.988980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:12.048409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 553/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:03:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:03:12.054333 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780ed4e-4584-4ff8-82d4-4e9462942bdf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:12.151165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780ed4e-4584-4ff8-82d4-4e9462942bdf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:12.185054 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb890e9-a5e1-4198-9411-296cab440c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:12.185674 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb890e9-a5e1-4198-9411-296cab440c75 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:12.185954 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb890e9-a5e1-4198-9411-296cab440c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:12.186227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb890e9-a5e1-4198-9411-296cab440c75 None admin] PATH_INFO: `/v3/projects/035c9529969d48f3899132d7b3bc15e0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:12.187361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb890e9-a5e1-4198-9411-296cab440c75 None admin] RBAC: Authorizing `identity:get_project(project_id=035c9529969d48f3899132d7b3bc15e0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:12.188793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb890e9-a5e1-4198-9411-296cab440c75 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:12.190409 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 563/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:03:12 2022] GET /identity/v3/projects/035c9529969d48f3899132d7b3bc15e0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:12.196456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a7c52-0928-4108-8073-a6086639501f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:12.293032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a7c52-0928-4108-8073-a6086639501f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:12.326370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b24f27-4879-4d43-9fa6-b82043195317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:12.326963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b24f27-4879-4d43-9fa6-b82043195317 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:12.327269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b24f27-4879-4d43-9fa6-b82043195317 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:12.327546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b24f27-4879-4d43-9fa6-b82043195317 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:12.328260 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b24f27-4879-4d43-9fa6-b82043195317 None admin] RBAC: Adding query filter params (role.id=0238ab71fb17471cb87e84097272c752, scope.project.id=035c9529969d48f3899132d7b3bc15e0, user.id=a44ff153d124495399c20f127f5c2d8c) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:12.328570 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b24f27-4879-4d43-9fa6-b82043195317 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:12.329994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b24f27-4879-4d43-9fa6-b82043195317 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:12.335179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 554/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:03:12 2022] GET /identity/v3/role_assignments?user.id=a44ff153d124495399c20f127f5c2d8c&scope.project.id=035c9529969d48f3899132d7b3bc15e0&role.id=0238ab71fb17471cb87e84097272c752 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:14.452776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7a2a9-e2e0-479a-9153-bab0bfac533d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:14.453336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7a2a9-e2e0-479a-9153-bab0bfac533d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:14.453336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7a2a9-e2e0-479a-9153-bab0bfac533d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:14.454643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 564/1118] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:14.459881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb15f8c-d7b5-4a6c-a83f-9db5647a332d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:14.460251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb15f8c-d7b5-4a6c-a83f-9db5647a332d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:14.460423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb15f8c-d7b5-4a6c-a83f-9db5647a332d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:14.467381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:14.480588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6eb15f8c-d7b5-4a6c-a83f-9db5647a332d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:14.481784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6eb15f8c-d7b5-4a6c-a83f-9db5647a332d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:14.546334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 555/1119] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:14.820078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7a2a9-e2e0-479a-9153-bab0bfac533d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:14.870392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:14.920442 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7a2a9-e2e0-479a-9153-bab0bfac533d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:14.953892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99985388-03a5-4614-8e39-44dabf025ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:14.954515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99985388-03a5-4614-8e39-44dabf025ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:14.954797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99985388-03a5-4614-8e39-44dabf025ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:14.955094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99985388-03a5-4614-8e39-44dabf025ca1 None admin] PATH_INFO: `/v3/roles/0238ab71fb17471cb87e84097272c752` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:14.956491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99985388-03a5-4614-8e39-44dabf025ca1 None admin] RBAC: Authorizing `identity:get_role(role_id=0238ab71fb17471cb87e84097272c752)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:14.957850 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99985388-03a5-4614-8e39-44dabf025ca1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:14.959188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 565/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:14 2022] GET /identity/v3/roles/0238ab71fb17471cb87e84097272c752 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:14.965979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb15f8c-d7b5-4a6c-a83f-9db5647a332d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:15.062705 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb15f8c-d7b5-4a6c-a83f-9db5647a332d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:15.092067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:15.097250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585757d4-e0f8-496d-8807-5cd22dcfa03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:15.097875 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585757d4-e0f8-496d-8807-5cd22dcfa03f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:15.098163 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585757d4-e0f8-496d-8807-5cd22dcfa03f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:15.098436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585757d4-e0f8-496d-8807-5cd22dcfa03f None admin] PATH_INFO: `/v3/users/a44ff153d124495399c20f127f5c2d8c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:15.099575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585757d4-e0f8-496d-8807-5cd22dcfa03f None admin] RBAC: Authorizing `identity:get_user(user_id=a44ff153d124495399c20f127f5c2d8c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:15.101010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585757d4-e0f8-496d-8807-5cd22dcfa03f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:15.102681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 556/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:14 2022] GET /identity/v3/users/a44ff153d124495399c20f127f5c2d8c => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:15.108500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99985388-03a5-4614-8e39-44dabf025ca1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:15.205634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99985388-03a5-4614-8e39-44dabf025ca1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:15.222713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:15.239439 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5685aa-9635-4864-9e46-32723be90ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:15.240066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5685aa-9635-4864-9e46-32723be90ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:15.240348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5685aa-9635-4864-9e46-32723be90ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:15.240625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5685aa-9635-4864-9e46-32723be90ac5 None admin] PATH_INFO: `/v3/projects/035c9529969d48f3899132d7b3bc15e0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:15.241763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5685aa-9635-4864-9e46-32723be90ac5 None admin] RBAC: Authorizing `identity:get_project(project_id=035c9529969d48f3899132d7b3bc15e0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:15.243312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5685aa-9635-4864-9e46-32723be90ac5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:15.244943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 566/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:03:15 2022] GET /identity/v3/projects/035c9529969d48f3899132d7b3bc15e0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:15.250790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585757d4-e0f8-496d-8807-5cd22dcfa03f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:15.348613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585757d4-e0f8-496d-8807-5cd22dcfa03f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:15.381574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69605cba-9f76-4396-909b-0a514a56b135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:15.382216 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69605cba-9f76-4396-909b-0a514a56b135 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:15.382492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69605cba-9f76-4396-909b-0a514a56b135 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:15.382759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69605cba-9f76-4396-909b-0a514a56b135 None admin] PATH_INFO: `/v3/projects/035c9529969d48f3899132d7b3bc15e0/users/a44ff153d124495399c20f127f5c2d8c/roles/0238ab71fb17471cb87e84097272c752` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:15.385469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69605cba-9f76-4396-909b-0a514a56b135 None admin] RBAC: Authorizing `identity:create_grant(project_id=035c9529969d48f3899132d7b3bc15e0, user_id=a44ff153d124495399c20f127f5c2d8c, role_id=0238ab71fb17471cb87e84097272c752)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:15.386887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69605cba-9f76-4396-909b-0a514a56b135 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:15.399945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 557/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:03:15 2022] PUT /identity/v3/projects/035c9529969d48f3899132d7b3bc15e0/users/a44ff153d124495399c20f127f5c2d8c/roles/0238ab71fb17471cb87e84097272c752 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:17.537166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edc890-7398-47e2-83ed-1c11241665c9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:17.537704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edc890-7398-47e2-83ed-1c11241665c9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:17.537704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edc890-7398-47e2-83ed-1c11241665c9 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:17.539099 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 567/1124] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:17.544921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9688b86-392b-42a3-b08c-49f338d13024 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:17.545350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9688b86-392b-42a3-b08c-49f338d13024 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:17.545466 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9688b86-392b-42a3-b08c-49f338d13024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:17.567995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d9688b86-392b-42a3-b08c-49f338d13024 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:17.569692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d9688b86-392b-42a3-b08c-49f338d13024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:17.642290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:17.657771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 558/1125] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:17.942804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edc890-7398-47e2-83ed-1c11241665c9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:18.059784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edc890-7398-47e2-83ed-1c11241665c9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:18.101312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44302f-3b85-422a-9feb-7386d490aff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:18.101979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44302f-3b85-422a-9feb-7386d490aff6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:18.102427 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44302f-3b85-422a-9feb-7386d490aff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:18.102814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44302f-3b85-422a-9feb-7386d490aff6 None admin] PATH_INFO: `/v3/roles/0238ab71fb17471cb87e84097272c752` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:18.104848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44302f-3b85-422a-9feb-7386d490aff6 None admin] RBAC: Authorizing `identity:get_role(role_id=0238ab71fb17471cb87e84097272c752)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:18.106212 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44302f-3b85-422a-9feb-7386d490aff6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:18.107587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 568/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:17 2022] GET /identity/v3/roles/0238ab71fb17471cb87e84097272c752 => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:18.113690 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9688b86-392b-42a3-b08c-49f338d13024 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:18.229308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9688b86-392b-42a3-b08c-49f338d13024 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:18.268618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db40171-a3ab-45fc-b983-52d7f5860ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:18.269256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db40171-a3ab-45fc-b983-52d7f5860ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:18.269619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db40171-a3ab-45fc-b983-52d7f5860ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:18.269973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db40171-a3ab-45fc-b983-52d7f5860ab3 None admin] PATH_INFO: `/v3/users/a44ff153d124495399c20f127f5c2d8c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:18.271561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db40171-a3ab-45fc-b983-52d7f5860ab3 None admin] RBAC: Authorizing `identity:get_user(user_id=a44ff153d124495399c20f127f5c2d8c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:18.273144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db40171-a3ab-45fc-b983-52d7f5860ab3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:18.275346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 559/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:18 2022] GET /identity/v3/users/a44ff153d124495399c20f127f5c2d8c => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:03:18.281965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44302f-3b85-422a-9feb-7386d490aff6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:18.351774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:18.403533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44302f-3b85-422a-9feb-7386d490aff6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:18.437941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcf971-f5f3-43d4-aafb-0181b3e4fee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:18.438540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf971-f5f3-43d4-aafb-0181b3e4fee5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:18.438814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf971-f5f3-43d4-aafb-0181b3e4fee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:18.439106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf971-f5f3-43d4-aafb-0181b3e4fee5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:18.503953 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcf971-f5f3-43d4-aafb-0181b3e4fee5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:18.505316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcf971-f5f3-43d4-aafb-0181b3e4fee5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:18.558023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:18.605496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 569/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:03:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 13:03:18.611532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db40171-a3ab-45fc-b983-52d7f5860ab3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:18.670062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:18.711938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db40171-a3ab-45fc-b983-52d7f5860ab3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:18.744932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed7547-2b2b-4994-aed9-b0bfffb66ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:18.745559 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed7547-2b2b-4994-aed9-b0bfffb66ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:18.745839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed7547-2b2b-4994-aed9-b0bfffb66ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:18.746114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed7547-2b2b-4994-aed9-b0bfffb66ab9 None admin] PATH_INFO: `/v3/projects/035c9529969d48f3899132d7b3bc15e0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:18.747250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed7547-2b2b-4994-aed9-b0bfffb66ab9 None admin] RBAC: Authorizing `identity:get_project(project_id=035c9529969d48f3899132d7b3bc15e0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:18.748683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed7547-2b2b-4994-aed9-b0bfffb66ab9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:18.750310 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 560/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:03:18 2022] GET /identity/v3/projects/035c9529969d48f3899132d7b3bc15e0 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:18.756570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcf971-f5f3-43d4-aafb-0181b3e4fee5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:18.852655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcf971-f5f3-43d4-aafb-0181b3e4fee5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:18.885386 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3ce02-72c7-489c-8585-bc490e7343fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:18.885976 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3ce02-72c7-489c-8585-bc490e7343fd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:18.886253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3ce02-72c7-489c-8585-bc490e7343fd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:18.886524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3ce02-72c7-489c-8585-bc490e7343fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:18.887268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3ce02-72c7-489c-8585-bc490e7343fd None admin] RBAC: Adding query filter params (role.id=0238ab71fb17471cb87e84097272c752, scope.project.id=035c9529969d48f3899132d7b3bc15e0, user.id=a44ff153d124495399c20f127f5c2d8c) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:18.887575 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3ce02-72c7-489c-8585-bc490e7343fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:18.888988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3ce02-72c7-489c-8585-bc490e7343fd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:18.894561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 570/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:03:18 2022] GET /identity/v3/role_assignments?user.id=a44ff153d124495399c20f127f5c2d8c&scope.project.id=035c9529969d48f3899132d7b3bc15e0&role.id=0238ab71fb17471cb87e84097272c752 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:21.044544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a080b0-3c14-4622-bc76-d7b959ecb568 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:21.045106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a080b0-3c14-4622-bc76-d7b959ecb568 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:21.045106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a080b0-3c14-4622-bc76-d7b959ecb568 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:21.046434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 561/1131] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:21.051864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898ac1b-e412-4c62-9e9b-32871d2992a5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:21.052286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898ac1b-e412-4c62-9e9b-32871d2992a5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:21.052399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898ac1b-e412-4c62-9e9b-32871d2992a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:21.071086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8898ac1b-e412-4c62-9e9b-32871d2992a5 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:21.072289 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8898ac1b-e412-4c62-9e9b-32871d2992a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:21.084860 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:21.136700 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 571/1132] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:21.406036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a080b0-3c14-4622-bc76-d7b959ecb568 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:21.504314 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a080b0-3c14-4622-bc76-d7b959ecb568 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:21.528913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:21.538020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe6192-66cb-423e-83df-79e864e26c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:21.538647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe6192-66cb-423e-83df-79e864e26c8e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:21.538928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe6192-66cb-423e-83df-79e864e26c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:21.539794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe6192-66cb-423e-83df-79e864e26c8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:21.541051 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe6192-66cb-423e-83df-79e864e26c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:21.542485 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe6192-66cb-423e-83df-79e864e26c8e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:21.544088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 562/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:03:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:21.549725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898ac1b-e412-4c62-9e9b-32871d2992a5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:21.646385 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898ac1b-e412-4c62-9e9b-32871d2992a5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:21.679419 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f386b-af87-4404-93ea-beb152b82857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:21.680055 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f386b-af87-4404-93ea-beb152b82857 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:21.680332 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f386b-af87-4404-93ea-beb152b82857 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:21.680604 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f386b-af87-4404-93ea-beb152b82857 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:21.681514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f386b-af87-4404-93ea-beb152b82857 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:21.682941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f386b-af87-4404-93ea-beb152b82857 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:21.703594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:21.712654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 572/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:03:21 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:23.923274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310db48f-7761-4a67-a94e-44602413e55c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:23.923779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310db48f-7761-4a67-a94e-44602413e55c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:23.923891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310db48f-7761-4a67-a94e-44602413e55c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:23.925168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 563/1135] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:23.930522 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816bb2-a6ea-4374-b237-a15991e2aa6b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:23.931232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816bb2-a6ea-4374-b237-a15991e2aa6b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:23.931370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816bb2-a6ea-4374-b237-a15991e2aa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:23.950718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d9816bb2-a6ea-4374-b237-a15991e2aa6b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:23.951944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d9816bb2-a6ea-4374-b237-a15991e2aa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:24.015944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 573/1136] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:24.296495 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310db48f-7761-4a67-a94e-44602413e55c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:24.397691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310db48f-7761-4a67-a94e-44602413e55c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:24.431975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40813cb4-ae00-4f9b-872f-66e234233cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:24.432591 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40813cb4-ae00-4f9b-872f-66e234233cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:24.432867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40813cb4-ae00-4f9b-872f-66e234233cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:24.433133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40813cb4-ae00-4f9b-872f-66e234233cb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:24.434224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40813cb4-ae00-4f9b-872f-66e234233cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:24.435676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40813cb4-ae00-4f9b-872f-66e234233cb8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:24.437227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 564/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:03:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:24.442965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9816bb2-a6ea-4374-b237-a15991e2aa6b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:24.538384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9816bb2-a6ea-4374-b237-a15991e2aa6b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:24.566657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:24.573504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3a3ec-37c5-43c6-80c4-470b79b0b0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:24.574094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3a3ec-37c5-43c6-80c4-470b79b0b0b0 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:24.574367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3a3ec-37c5-43c6-80c4-470b79b0b0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:24.574633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3a3ec-37c5-43c6-80c4-470b79b0b0b0 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:24.575314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3a3ec-37c5-43c6-80c4-470b79b0b0b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:24.576723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3a3ec-37c5-43c6-80c4-470b79b0b0b0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:24.598613 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 574/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:03:24 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:26.815607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d25d2-8834-4d0f-a66b-c5aa404d0e6c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:26.816157 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d25d2-8834-4d0f-a66b-c5aa404d0e6c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:26.816157 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d25d2-8834-4d0f-a66b-c5aa404d0e6c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:26.817463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 565/1139] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:26.823552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2f67a-ba31-4d5f-8f45-367c65064275 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:26.823947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2f67a-ba31-4d5f-8f45-367c65064275 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:26.824103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2f67a-ba31-4d5f-8f45-367c65064275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:26.842501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e9a2f67a-ba31-4d5f-8f45-367c65064275 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:26.843720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e9a2f67a-ba31-4d5f-8f45-367c65064275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:26.908342 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 575/1140] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:27.188283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d25d2-8834-4d0f-a66b-c5aa404d0e6c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:27.249428 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:27.289574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d25d2-8834-4d0f-a66b-c5aa404d0e6c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:27.322920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c64360-d5e9-4c03-b018-4606b43b4b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:27.323566 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c64360-d5e9-4c03-b018-4606b43b4b7d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:27.323839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c64360-d5e9-4c03-b018-4606b43b4b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:27.324107 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c64360-d5e9-4c03-b018-4606b43b4b7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:27.333833 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c64360-d5e9-4c03-b018-4606b43b4b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-53c64360-d5e9-4c03-b018-4606b43b4b7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:03:27.339563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:27.340845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:27.342027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:27.342027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:27.342027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:27.342027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:27.342027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:03:27.342027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:03:27.342027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:03:27.342027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:03:27.342027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:03:27.342027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:03:27.342027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:27.342027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:03:27.342027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 566/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:03:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:27.346669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2f67a-ba31-4d5f-8f45-367c65064275 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:27.442983 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2f67a-ba31-4d5f-8f45-367c65064275 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:27.458651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:27.478294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdfd6e-d86e-4b90-9262-725cdc1991c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:27.478894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdfd6e-d86e-4b90-9262-725cdc1991c4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:27.479206 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdfd6e-d86e-4b90-9262-725cdc1991c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:27.479480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdfd6e-d86e-4b90-9262-725cdc1991c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:27.480139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdfd6e-d86e-4b90-9262-725cdc1991c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:27.480441 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdfd6e-d86e-4b90-9262-725cdc1991c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:27.481797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdfd6e-d86e-4b90-9262-725cdc1991c4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:27.489563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 576/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:03:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:03:27.495479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c64360-d5e9-4c03-b018-4606b43b4b7d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:27.593167 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c64360-d5e9-4c03-b018-4606b43b4b7d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:27.615946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:27.628223 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45045c-b061-45f7-9e53-860446a505c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:27.628851 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45045c-b061-45f7-9e53-860446a505c4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:27.629121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45045c-b061-45f7-9e53-860446a505c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:27.629388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45045c-b061-45f7-9e53-860446a505c4 None admin] PATH_INFO: `/v3/users/c51f8077467645a698d1976db3623e7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:27.643130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45045c-b061-45f7-9e53-860446a505c4 None admin] RBAC: Authorizing `identity:get_user(user_id=c51f8077467645a698d1976db3623e7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:27.644679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45045c-b061-45f7-9e53-860446a505c4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:27.646278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 567/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:27 2022] GET /identity/v3/users/c51f8077467645a698d1976db3623e7a => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:27.652459 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdfd6e-d86e-4b90-9262-725cdc1991c4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:27.748951 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdfd6e-d86e-4b90-9262-725cdc1991c4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:27.781694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafed068-14ad-4a5a-9cf1-423d34f5ee4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:27.782284 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafed068-14ad-4a5a-9cf1-423d34f5ee4f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:27.782559 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafed068-14ad-4a5a-9cf1-423d34f5ee4f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:27.782828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafed068-14ad-4a5a-9cf1-423d34f5ee4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:27.826928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:27.848588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafed068-14ad-4a5a-9cf1-423d34f5ee4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:27.850012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafed068-14ad-4a5a-9cf1-423d34f5ee4f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:27.943726 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 577/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:03:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:03:27.950428 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45045c-b061-45f7-9e53-860446a505c4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:28.047280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45045c-b061-45f7-9e53-860446a505c4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:28.080149 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7fa7e-51ef-46e3-a5cc-8109424aed8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:28.080766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7fa7e-51ef-46e3-a5cc-8109424aed8e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:28.081043 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7fa7e-51ef-46e3-a5cc-8109424aed8e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:28.081340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7fa7e-51ef-46e3-a5cc-8109424aed8e None admin] PATH_INFO: `/v3/projects/08238632b0ad4985b58cd1f750a5f89d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:28.082440 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7fa7e-51ef-46e3-a5cc-8109424aed8e None admin] RBAC: Authorizing `identity:get_project(project_id=08238632b0ad4985b58cd1f750a5f89d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:28.083888 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7fa7e-51ef-46e3-a5cc-8109424aed8e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:28.086047 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 568/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:03:27 2022] GET /identity/v3/projects/08238632b0ad4985b58cd1f750a5f89d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:28.092266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafed068-14ad-4a5a-9cf1-423d34f5ee4f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:28.173952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:28.194940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafed068-14ad-4a5a-9cf1-423d34f5ee4f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:28.227693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffe98d-3d9a-4dc0-be22-be47b205148c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:28.228273 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffe98d-3d9a-4dc0-be22-be47b205148c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:28.228545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffe98d-3d9a-4dc0-be22-be47b205148c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:28.228808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffe98d-3d9a-4dc0-be22-be47b205148c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:28.229515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffe98d-3d9a-4dc0-be22-be47b205148c None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=08238632b0ad4985b58cd1f750a5f89d, user.id=c51f8077467645a698d1976db3623e7a) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:28.229816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffe98d-3d9a-4dc0-be22-be47b205148c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:28.231242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffe98d-3d9a-4dc0-be22-be47b205148c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:28.236377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 578/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:03:28 2022] GET /identity/v3/role_assignments?user.id=c51f8077467645a698d1976db3623e7a&scope.project.id=08238632b0ad4985b58cd1f750a5f89d&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:30.343796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab23e0-9f16-4bc4-864a-cd056cd72e41 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:30.344353 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab23e0-9f16-4bc4-864a-cd056cd72e41 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:30.344353 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab23e0-9f16-4bc4-864a-cd056cd72e41 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:30.345681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 569/1147] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:30.351063 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91b00f-8bcb-46ab-a6e7-302251c168f2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:30.351468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91b00f-8bcb-46ab-a6e7-302251c168f2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:30.351619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91b00f-8bcb-46ab-a6e7-302251c168f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:30.369912 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-db91b00f-8bcb-46ab-a6e7-302251c168f2 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:30.371139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-db91b00f-8bcb-46ab-a6e7-302251c168f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:30.436162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 579/1148] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:30.704509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab23e0-9f16-4bc4-864a-cd056cd72e41 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:30.753465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:30.803303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab23e0-9f16-4bc4-864a-cd056cd72e41 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:30.836099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ce2d6-6216-4a86-b94c-725660546797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:30.836712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ce2d6-6216-4a86-b94c-725660546797 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:30.836989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ce2d6-6216-4a86-b94c-725660546797 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:30.837260 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ce2d6-6216-4a86-b94c-725660546797 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:30.846959 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ce2d6-6216-4a86-b94c-725660546797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e66ce2d6-6216-4a86-b94c-725660546797 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:03:30.852404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:30.853546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:30.854798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:30.854798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:30.854798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:30.854798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:30.854798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:03:30.854798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:03:30.854798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:03:30.854798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:03:30.854798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:03:30.854798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:03:30.854798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:30.854798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:03:30.854798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 570/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:03:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:30.859461 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91b00f-8bcb-46ab-a6e7-302251c168f2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:30.926562 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:30.955542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91b00f-8bcb-46ab-a6e7-302251c168f2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:30.988911 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f529b4b-e4a4-47b6-a123-9bd712fdde05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:30.989507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f529b4b-e4a4-47b6-a123-9bd712fdde05 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:30.989787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f529b4b-e4a4-47b6-a123-9bd712fdde05 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:30.990058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f529b4b-e4a4-47b6-a123-9bd712fdde05 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:30.990711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f529b4b-e4a4-47b6-a123-9bd712fdde05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:30.991013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f529b4b-e4a4-47b6-a123-9bd712fdde05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:30.992552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f529b4b-e4a4-47b6-a123-9bd712fdde05 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:31.000469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 580/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:03:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:31.006247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ce2d6-6216-4a86-b94c-725660546797 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:31.104790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ce2d6-6216-4a86-b94c-725660546797 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:31.113261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:31.138714 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c209b-148e-4403-adda-58c14139d91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:31.139364 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c209b-148e-4403-adda-58c14139d91e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:31.139642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c209b-148e-4403-adda-58c14139d91e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:31.139912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c209b-148e-4403-adda-58c14139d91e None admin] PATH_INFO: `/v3/users/c51f8077467645a698d1976db3623e7a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:31.141017 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c209b-148e-4403-adda-58c14139d91e None admin] RBAC: Authorizing `identity:get_user(user_id=c51f8077467645a698d1976db3623e7a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:31.142445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c209b-148e-4403-adda-58c14139d91e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:31.144037 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 571/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:31 2022] GET /identity/v3/users/c51f8077467645a698d1976db3623e7a => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:31.149910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f529b4b-e4a4-47b6-a123-9bd712fdde05 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:31.248058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f529b4b-e4a4-47b6-a123-9bd712fdde05 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:31.280882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586db87c-fc36-4a93-a90d-d287005b08a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:31.281503 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586db87c-fc36-4a93-a90d-d287005b08a9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:31.281783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586db87c-fc36-4a93-a90d-d287005b08a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:31.282053 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586db87c-fc36-4a93-a90d-d287005b08a9 None admin] PATH_INFO: `/v3/projects/08238632b0ad4985b58cd1f750a5f89d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:31.283187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586db87c-fc36-4a93-a90d-d287005b08a9 None admin] RBAC: Authorizing `identity:get_project(project_id=08238632b0ad4985b58cd1f750a5f89d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:31.284609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586db87c-fc36-4a93-a90d-d287005b08a9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:31.286252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 581/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:03:31 2022] GET /identity/v3/projects/08238632b0ad4985b58cd1f750a5f89d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:31.292150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c209b-148e-4403-adda-58c14139d91e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:31.389088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c209b-148e-4403-adda-58c14139d91e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:31.420661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a00bd3-cf77-49aa-b952-10353928bb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:31.421299 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a00bd3-cf77-49aa-b952-10353928bb04 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:31.421573 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a00bd3-cf77-49aa-b952-10353928bb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:31.421844 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a00bd3-cf77-49aa-b952-10353928bb04 None admin] PATH_INFO: `/v3/projects/08238632b0ad4985b58cd1f750a5f89d/users/c51f8077467645a698d1976db3623e7a/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:31.423700 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a00bd3-cf77-49aa-b952-10353928bb04 None admin] RBAC: Authorizing `identity:create_grant(project_id=08238632b0ad4985b58cd1f750a5f89d, user_id=c51f8077467645a698d1976db3623e7a, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:31.425123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a00bd3-cf77-49aa-b952-10353928bb04 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:31.438532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 572/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:03:31 2022] PUT /identity/v3/projects/08238632b0ad4985b58cd1f750a5f89d/users/c51f8077467645a698d1976db3623e7a/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:33.561331 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffa2e1-18db-4b64-b0de-4ac5df3b23aa None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:33.561985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffa2e1-18db-4b64-b0de-4ac5df3b23aa None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:33.561985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffa2e1-18db-4b64-b0de-4ac5df3b23aa None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:33.563252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 582/1154] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:33.568469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6ffa0-092b-44aa-acb3-ae74769f7dda None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:33.569517 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6ffa0-092b-44aa-acb3-ae74769f7dda None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:33.569628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6ffa0-092b-44aa-acb3-ae74769f7dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:33.587716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7fb6ffa0-092b-44aa-acb3-ae74769f7dda None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:33.588918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7fb6ffa0-092b-44aa-acb3-ae74769f7dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:33.617979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:33.659770 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 573/1155] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:33.925475 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffa2e1-18db-4b64-b0de-4ac5df3b23aa None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:33.928947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:34.028205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffa2e1-18db-4b64-b0de-4ac5df3b23aa None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:34.060576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96450995-e2df-4799-9d0b-9bb831e52a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:34.061192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96450995-e2df-4799-9d0b-9bb831e52a28 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:34.061470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96450995-e2df-4799-9d0b-9bb831e52a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:34.061741 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96450995-e2df-4799-9d0b-9bb831e52a28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:34.072477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96450995-e2df-4799-9d0b-9bb831e52a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-96450995-e2df-4799-9d0b-9bb831e52a28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:03:34.077167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:34.078400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:34.079487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:34.079487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:34.079487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:34.079487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:34.079487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:03:34.079487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:03:34.079487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:03:34.079487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:03:34.079487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:03:34.079487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:03:34.079487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:34.079487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:03:34.079487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 583/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:03:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:34.084412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6ffa0-092b-44aa-acb3-ae74769f7dda None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:34.187366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6ffa0-092b-44aa-acb3-ae74769f7dda None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:34.221236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e3425-82ea-4a20-b3da-6f3a7f6ff1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:34.221894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e3425-82ea-4a20-b3da-6f3a7f6ff1ad None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:34.222175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e3425-82ea-4a20-b3da-6f3a7f6ff1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:34.222445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e3425-82ea-4a20-b3da-6f3a7f6ff1ad None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:34.223139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e3425-82ea-4a20-b3da-6f3a7f6ff1ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:34.223444 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e3425-82ea-4a20-b3da-6f3a7f6ff1ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:34.224806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e3425-82ea-4a20-b3da-6f3a7f6ff1ad None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:34.232670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 574/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:03:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:34.238562 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96450995-e2df-4799-9d0b-9bb831e52a28 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:34.300353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:34.338419 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96450995-e2df-4799-9d0b-9bb831e52a28 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:34.370886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4c3d4-4287-43ab-ab21-31ef3af8889c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:34.371532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4c3d4-4287-43ab-ab21-31ef3af8889c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:34.371811 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4c3d4-4287-43ab-ab21-31ef3af8889c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:34.372084 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4c3d4-4287-43ab-ab21-31ef3af8889c None admin] PATH_INFO: `/v3/users/c51f8077467645a698d1976db3623e7a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:34.373185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4c3d4-4287-43ab-ab21-31ef3af8889c None admin] RBAC: Authorizing `identity:get_user(user_id=c51f8077467645a698d1976db3623e7a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:34.374604 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4c3d4-4287-43ab-ab21-31ef3af8889c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:34.376200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 584/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:34 2022] GET /identity/v3/users/c51f8077467645a698d1976db3623e7a => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:34.382118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e3425-82ea-4a20-b3da-6f3a7f6ff1ad None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:34.410067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:34.480849 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e3425-82ea-4a20-b3da-6f3a7f6ff1ad None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:34.513062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c0f0e-90bd-4d2b-945b-dda826426cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:34.513660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0f0e-90bd-4d2b-945b-dda826426cde None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:34.513937 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0f0e-90bd-4d2b-945b-dda826426cde None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:34.514206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0f0e-90bd-4d2b-945b-dda826426cde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:34.578746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c0f0e-90bd-4d2b-945b-dda826426cde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:34.580235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c0f0e-90bd-4d2b-945b-dda826426cde None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:34.606813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:34.679073 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 575/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:03:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:03:34.685419 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4c3d4-4287-43ab-ab21-31ef3af8889c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:34.781020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4c3d4-4287-43ab-ab21-31ef3af8889c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:34.808944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:34.814894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fae7b0-07b8-4c19-99bf-1df2d87faa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:34.815544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fae7b0-07b8-4c19-99bf-1df2d87faa88 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:34.815822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fae7b0-07b8-4c19-99bf-1df2d87faa88 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:34.816094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fae7b0-07b8-4c19-99bf-1df2d87faa88 None admin] PATH_INFO: `/v3/projects/08238632b0ad4985b58cd1f750a5f89d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:34.817194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fae7b0-07b8-4c19-99bf-1df2d87faa88 None admin] RBAC: Authorizing `identity:get_project(project_id=08238632b0ad4985b58cd1f750a5f89d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:34.818704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fae7b0-07b8-4c19-99bf-1df2d87faa88 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:34.820564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 585/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:03:34 2022] GET /identity/v3/projects/08238632b0ad4985b58cd1f750a5f89d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:34.826760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c0f0e-90bd-4d2b-945b-dda826426cde None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:34.923361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c0f0e-90bd-4d2b-945b-dda826426cde None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:34.949086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:34.957008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45468da1-b5f1-4848-8dc0-57c6f3d140fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:34.957602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45468da1-b5f1-4848-8dc0-57c6f3d140fe None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:34.957881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45468da1-b5f1-4848-8dc0-57c6f3d140fe None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:34.958154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45468da1-b5f1-4848-8dc0-57c6f3d140fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:34.958869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45468da1-b5f1-4848-8dc0-57c6f3d140fe None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=08238632b0ad4985b58cd1f750a5f89d, user.id=c51f8077467645a698d1976db3623e7a) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:34.959212 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45468da1-b5f1-4848-8dc0-57c6f3d140fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:34.961276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45468da1-b5f1-4848-8dc0-57c6f3d140fe None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:34.966893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 576/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:03:34 2022] GET /identity/v3/role_assignments?user.id=c51f8077467645a698d1976db3623e7a&scope.project.id=08238632b0ad4985b58cd1f750a5f89d&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:37.112942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d755d5-7157-42a6-854c-8a885c5c21eb None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:37.113497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d755d5-7157-42a6-854c-8a885c5c21eb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:37.113497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d755d5-7157-42a6-854c-8a885c5c21eb None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:37.114832 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 586/1162] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:37.120654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b84ae-2f0f-4d30-a66b-afaf7a4918ee None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:37.121073 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b84ae-2f0f-4d30-a66b-afaf7a4918ee None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:37.121201 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b84ae-2f0f-4d30-a66b-afaf7a4918ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:37.139501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bb0b84ae-2f0f-4d30-a66b-afaf7a4918ee None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:37.140680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bb0b84ae-2f0f-4d30-a66b-afaf7a4918ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:37.204938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 577/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:37.473033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d755d5-7157-42a6-854c-8a885c5c21eb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:37.572245 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d755d5-7157-42a6-854c-8a885c5c21eb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:37.605434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1602f-adb8-4cee-8499-cc9e2c032e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:37.606048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1602f-adb8-4cee-8499-cc9e2c032e51 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:37.606325 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1602f-adb8-4cee-8499-cc9e2c032e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:37.606600 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1602f-adb8-4cee-8499-cc9e2c032e51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:37.669154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:37.673315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1602f-adb8-4cee-8499-cc9e2c032e51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:37.674764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1602f-adb8-4cee-8499-cc9e2c032e51 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:37.770089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 587/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:03:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:03:37.776932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b84ae-2f0f-4d30-a66b-afaf7a4918ee None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:37.871335 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b84ae-2f0f-4d30-a66b-afaf7a4918ee None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:37.893231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:37.904711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8acca-19b2-448b-a52f-d8e8843d8842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:37.905328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8acca-19b2-448b-a52f-d8e8843d8842 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:37.905598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8acca-19b2-448b-a52f-d8e8843d8842 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:37.905864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8acca-19b2-448b-a52f-d8e8843d8842 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:37.912362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8acca-19b2-448b-a52f-d8e8843d8842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:37.913779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8acca-19b2-448b-a52f-d8e8843d8842 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f1a8acca-19b2-448b-a52f-d8e8843d8842 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:37.924735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:37.925981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:03:37.927273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 578/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 13:03:37 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:37.932601 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1602f-adb8-4cee-8499-cc9e2c032e51 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:38.025173 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:38.029081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1602f-adb8-4cee-8499-cc9e2c032e51 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:38.062549 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f10603-c610-45d9-ad74-c9c0b9554ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:38.063169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f10603-c610-45d9-ad74-c9c0b9554ede None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:38.063447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f10603-c610-45d9-ad74-c9c0b9554ede None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:38.063718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f10603-c610-45d9-ad74-c9c0b9554ede None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:38.064346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f10603-c610-45d9-ad74-c9c0b9554ede None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:38.064651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f10603-c610-45d9-ad74-c9c0b9554ede None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:38.065994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f10603-c610-45d9-ad74-c9c0b9554ede None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:38.073442 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 588/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 9 13:03:37 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:40.206355 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbb8e3-e1f5-47d6-b98a-06dade6890c5 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:40.206886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbb8e3-e1f5-47d6-b98a-06dade6890c5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:40.206886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbb8e3-e1f5-47d6-b98a-06dade6890c5 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:40.208283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 579/1167] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:40.213873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f37f8a-0ad9-4cbe-8cb6-925ba5a092f7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:40.214277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f37f8a-0ad9-4cbe-8cb6-925ba5a092f7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:40.214422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f37f8a-0ad9-4cbe-8cb6-925ba5a092f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:40.237643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d6f37f8a-0ad9-4cbe-8cb6-925ba5a092f7 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:40.238834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d6f37f8a-0ad9-4cbe-8cb6-925ba5a092f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:40.306081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 589/1168] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:40.575235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbb8e3-e1f5-47d6-b98a-06dade6890c5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:40.674015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbb8e3-e1f5-47d6-b98a-06dade6890c5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:40.707381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7dfc21-9c37-4268-9c5c-e1bb22fb756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:40.707991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dfc21-9c37-4268-9c5c-e1bb22fb756e None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:40.708273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dfc21-9c37-4268-9c5c-e1bb22fb756e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:40.708548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dfc21-9c37-4268-9c5c-e1bb22fb756e None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:40.709850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7dfc21-9c37-4268-9c5c-e1bb22fb756e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:40.711249 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7dfc21-9c37-4268-9c5c-e1bb22fb756e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:40.726831 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 580/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Fri Sep 9 13:03:40 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:42.903307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fa3c7-e166-4beb-864f-77115f90e56f None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:42.903820 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fa3c7-e166-4beb-864f-77115f90e56f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:42.903820 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fa3c7-e166-4beb-864f-77115f90e56f None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:42.905107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 590/1170] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:42.910447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3319cf-72df-4642-a56b-424f0aac5766 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:42.910824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3319cf-72df-4642-a56b-424f0aac5766 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:42.910996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3319cf-72df-4642-a56b-424f0aac5766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:42.930248 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cf3319cf-72df-4642-a56b-424f0aac5766 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:42.931488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cf3319cf-72df-4642-a56b-424f0aac5766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:42.965008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:42.997324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 581/1171] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:43.270719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8fa3c7-e166-4beb-864f-77115f90e56f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:43.361022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:43.370664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8fa3c7-e166-4beb-864f-77115f90e56f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:43.403665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc6eea-fc9c-4fc9-af95-da537573d2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:43.404286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc6eea-fc9c-4fc9-af95-da537573d2ba None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:43.404566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc6eea-fc9c-4fc9-af95-da537573d2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:43.404867 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc6eea-fc9c-4fc9-af95-da537573d2ba None admin] PATH_INFO: `/v3/domains/209d4b04ba0b4458ab434fb91dabfab9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:43.415653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc6eea-fc9c-4fc9-af95-da537573d2ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=209d4b04ba0b4458ab434fb91dabfab9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:43.417086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc6eea-fc9c-4fc9-af95-da537573d2ba None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:43.418655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 591/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Fri Sep 9 13:03:43 2022] GET /identity/v3/domains/209d4b04ba0b4458ab434fb91dabfab9 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:43.424426 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3319cf-72df-4642-a56b-424f0aac5766 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:43.525375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3319cf-72df-4642-a56b-424f0aac5766 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:43.557874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88736fa6-ba79-455d-bd6c-01c667935dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:43.558465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88736fa6-ba79-455d-bd6c-01c667935dc7 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:43.558739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88736fa6-ba79-455d-bd6c-01c667935dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:43.559043 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88736fa6-ba79-455d-bd6c-01c667935dc7 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:43.559711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88736fa6-ba79-455d-bd6c-01c667935dc7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:43.561132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88736fa6-ba79-455d-bd6c-01c667935dc7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:43.588192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 582/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:03:43 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:45.766175 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719cde8-95c9-4af7-b3fb-a58c19c2639e None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:45.766722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719cde8-95c9-4af7-b3fb-a58c19c2639e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:45.766722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719cde8-95c9-4af7-b3fb-a58c19c2639e None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:45.768081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 592/1174] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:45.774315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194a385-dd4c-4756-889c-d66090b28977 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:45.774736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194a385-dd4c-4756-889c-d66090b28977 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:45.774837 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194a385-dd4c-4756-889c-d66090b28977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:45.792922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f194a385-dd4c-4756-889c-d66090b28977 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:45.794089 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f194a385-dd4c-4756-889c-d66090b28977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:45.811166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:45.860818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 583/1175] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:46.135067 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719cde8-95c9-4af7-b3fb-a58c19c2639e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:46.233501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719cde8-95c9-4af7-b3fb-a58c19c2639e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:46.266758 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d19014-e36c-4ad7-b386-09262545e455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:46.267396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d19014-e36c-4ad7-b386-09262545e455 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:46.267674 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d19014-e36c-4ad7-b386-09262545e455 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:46.267945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d19014-e36c-4ad7-b386-09262545e455 None admin] PATH_INFO: `/v3/domains/209d4b04ba0b4458ab434fb91dabfab9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:46.269079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d19014-e36c-4ad7-b386-09262545e455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=209d4b04ba0b4458ab434fb91dabfab9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:46.270494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d19014-e36c-4ad7-b386-09262545e455 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:46.272668 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 593/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Fri Sep 9 13:03:46 2022] GET /identity/v3/domains/209d4b04ba0b4458ab434fb91dabfab9 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:46.278420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194a385-dd4c-4756-889c-d66090b28977 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:46.374281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194a385-dd4c-4756-889c-d66090b28977 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:46.406965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f42b2f-f01d-4008-883a-ae00d9c53bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:46.407540 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f42b2f-f01d-4008-883a-ae00d9c53bd7 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:46.407817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f42b2f-f01d-4008-883a-ae00d9c53bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:46.408085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f42b2f-f01d-4008-883a-ae00d9c53bd7 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:46.408722 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f42b2f-f01d-4008-883a-ae00d9c53bd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:46.410145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f42b2f-f01d-4008-883a-ae00d9c53bd7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:46.431406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 584/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:03:46 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:48.594462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f11766-3950-435f-9e8d-c5cd7b472939 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:48.595014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f11766-3950-435f-9e8d-c5cd7b472939 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:48.595014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f11766-3950-435f-9e8d-c5cd7b472939 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:48.596358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 594/1178] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:48.602095 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8f9f7-8965-4765-96a4-e1ceff508a78 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:48.602532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8f9f7-8965-4765-96a4-e1ceff508a78 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:48.602639 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8f9f7-8965-4765-96a4-e1ceff508a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:48.621004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e0c8f9f7-8965-4765-96a4-e1ceff508a78 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:48.622187 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e0c8f9f7-8965-4765-96a4-e1ceff508a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:48.624926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:48.688535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 585/1179] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:48.961585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f11766-3950-435f-9e8d-c5cd7b472939 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:48.984144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:49.063350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f11766-3950-435f-9e8d-c5cd7b472939 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:49.095825 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbd88c-f439-4925-aa2e-cc0bea1bead3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:49.096445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbd88c-f439-4925-aa2e-cc0bea1bead3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:49.096722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbd88c-f439-4925-aa2e-cc0bea1bead3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:49.096987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbd88c-f439-4925-aa2e-cc0bea1bead3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:49.107940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbd88c-f439-4925-aa2e-cc0bea1bead3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-49fbd88c-f439-4925-aa2e-cc0bea1bead3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:03:49.112587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:49.113731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:49.114884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:49.114884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:49.114884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:49.114884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:49.114884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:03:49.114884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:03:49.114884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:03:49.114884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:03:49.114884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:03:49.114884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:03:49.114884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:49.114884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:03:49.114884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 595/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:03:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:49.119779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8f9f7-8965-4765-96a4-e1ceff508a78 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:49.216738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8f9f7-8965-4765-96a4-e1ceff508a78 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:49.250125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c6e5a-b243-4edf-a2c1-1751d920dad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:49.250726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c6e5a-b243-4edf-a2c1-1751d920dad6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:49.251006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c6e5a-b243-4edf-a2c1-1751d920dad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:49.251312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c6e5a-b243-4edf-a2c1-1751d920dad6 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:49.251979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c6e5a-b243-4edf-a2c1-1751d920dad6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:49.252285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c6e5a-b243-4edf-a2c1-1751d920dad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:49.253655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c6e5a-b243-4edf-a2c1-1751d920dad6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:49.262063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 586/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:03:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:49.268124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbd88c-f439-4925-aa2e-cc0bea1bead3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:49.347008 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:49.366224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbd88c-f439-4925-aa2e-cc0bea1bead3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:49.398651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3da26c-cb7a-4f31-a7b4-e07323577c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:49.399296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3da26c-cb7a-4f31-a7b4-e07323577c43 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:49.399579 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3da26c-cb7a-4f31-a7b4-e07323577c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:49.399852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3da26c-cb7a-4f31-a7b4-e07323577c43 None admin] PATH_INFO: `/v3/users/bd9092501ef64b55b79e57afb8586f0d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:49.412653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3da26c-cb7a-4f31-a7b4-e07323577c43 None admin] RBAC: Authorizing `identity:get_user(user_id=bd9092501ef64b55b79e57afb8586f0d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:49.414077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3da26c-cb7a-4f31-a7b4-e07323577c43 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:49.415728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 596/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:49 2022] GET /identity/v3/users/bd9092501ef64b55b79e57afb8586f0d => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:49.421478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c6e5a-b243-4edf-a2c1-1751d920dad6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:49.424469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:49.521166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c6e5a-b243-4edf-a2c1-1751d920dad6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:49.554738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3b34d-4eda-430b-9ab3-667d5185ec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:49.555362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3b34d-4eda-430b-9ab3-667d5185ec98 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:49.555637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3b34d-4eda-430b-9ab3-667d5185ec98 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:49.555905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3b34d-4eda-430b-9ab3-667d5185ec98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:49.621604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3b34d-4eda-430b-9ab3-667d5185ec98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:49.623067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3b34d-4eda-430b-9ab3-667d5185ec98 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:49.624828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:49.719556 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 587/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:03:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:03:49.725882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3da26c-cb7a-4f31-a7b4-e07323577c43 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:49.822779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3da26c-cb7a-4f31-a7b4-e07323577c43 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:49.856759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1d968-a543-4c3b-8ec6-6a7ce76461f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:49.857368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d968-a543-4c3b-8ec6-6a7ce76461f4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:49.857644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d968-a543-4c3b-8ec6-6a7ce76461f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:49.857916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d968-a543-4c3b-8ec6-6a7ce76461f4 None admin] PATH_INFO: `/v3/projects/a00952d9403b42528c225d5daf466667` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:49.859018 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1d968-a543-4c3b-8ec6-6a7ce76461f4 None admin] RBAC: Authorizing `identity:get_project(project_id=a00952d9403b42528c225d5daf466667)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:49.860463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1d968-a543-4c3b-8ec6-6a7ce76461f4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:49.862066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 597/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:03:49 2022] GET /identity/v3/projects/a00952d9403b42528c225d5daf466667 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:49.868579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3b34d-4eda-430b-9ab3-667d5185ec98 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:49.964946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3b34d-4eda-430b-9ab3-667d5185ec98 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:49.966440 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:49.999540 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5d3fe-7ab3-4459-87d6-1d0d37027a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:50.000132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5d3fe-7ab3-4459-87d6-1d0d37027a12 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:50.000412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5d3fe-7ab3-4459-87d6-1d0d37027a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:50.000685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5d3fe-7ab3-4459-87d6-1d0d37027a12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:50.001397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5d3fe-7ab3-4459-87d6-1d0d37027a12 None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=a00952d9403b42528c225d5daf466667, user.id=bd9092501ef64b55b79e57afb8586f0d) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:50.001706 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5d3fe-7ab3-4459-87d6-1d0d37027a12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:50.003161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5d3fe-7ab3-4459-87d6-1d0d37027a12 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:50.008874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 588/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:03:49 2022] GET /identity/v3/role_assignments?user.id=bd9092501ef64b55b79e57afb8586f0d&scope.project.id=a00952d9403b42528c225d5daf466667&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:52.156235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1562d-f194-4494-9edd-073a258ba375 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:52.156785 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1562d-f194-4494-9edd-073a258ba375 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:52.156785 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1562d-f194-4494-9edd-073a258ba375 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:52.158096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 598/1186] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:52.164079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043f875-9c24-4a32-9259-e426076b5f53 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:52.164453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043f875-9c24-4a32-9259-e426076b5f53 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:52.164623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043f875-9c24-4a32-9259-e426076b5f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:52.182897 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8043f875-9c24-4a32-9259-e426076b5f53 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:52.184121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8043f875-9c24-4a32-9259-e426076b5f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:52.248539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 589/1187] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:03:52.518485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1562d-f194-4494-9edd-073a258ba375 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:52.527869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:52.618328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1562d-f194-4494-9edd-073a258ba375 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:52.652080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee54367-6c23-4b99-af76-85144cd60bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:52.652689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee54367-6c23-4b99-af76-85144cd60bef None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:52.652963 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee54367-6c23-4b99-af76-85144cd60bef None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:52.653231 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee54367-6c23-4b99-af76-85144cd60bef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:52.663744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee54367-6c23-4b99-af76-85144cd60bef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-fee54367-6c23-4b99-af76-85144cd60bef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:03:52.668345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:52.669451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:52.670350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:52.670350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:52.670350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:52.670350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:52.670350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:03:52.670350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:03:52.670350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:03:52.670350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:03:52.670350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:03:52.670350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:03:52.670350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:52.670350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:03:52.670350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 599/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:03:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:52.675579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043f875-9c24-4a32-9259-e426076b5f53 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:52.767181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:52.773987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043f875-9c24-4a32-9259-e426076b5f53 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:52.808118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618eb50-f123-43b4-8bdb-1ee01b3692b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:52.808732 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618eb50-f123-43b4-8bdb-1ee01b3692b8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:52.809012 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618eb50-f123-43b4-8bdb-1ee01b3692b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:52.809286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618eb50-f123-43b4-8bdb-1ee01b3692b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:52.809943 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618eb50-f123-43b4-8bdb-1ee01b3692b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:52.810250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618eb50-f123-43b4-8bdb-1ee01b3692b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:52.811649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618eb50-f123-43b4-8bdb-1ee01b3692b8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:52.820450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 590/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:03:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:52.826181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee54367-6c23-4b99-af76-85144cd60bef None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:52.891394 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:52.924737 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee54367-6c23-4b99-af76-85144cd60bef None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:52.958303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86694f53-153d-447e-b642-4a7498aa13d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:52.958913 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86694f53-153d-447e-b642-4a7498aa13d9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:52.959226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86694f53-153d-447e-b642-4a7498aa13d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:52.959501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86694f53-153d-447e-b642-4a7498aa13d9 None admin] PATH_INFO: `/v3/users/bd9092501ef64b55b79e57afb8586f0d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:52.960667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86694f53-153d-447e-b642-4a7498aa13d9 None admin] RBAC: Authorizing `identity:get_user(user_id=bd9092501ef64b55b79e57afb8586f0d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:52.962091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86694f53-153d-447e-b642-4a7498aa13d9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:52.963684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 600/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:52 2022] GET /identity/v3/users/bd9092501ef64b55b79e57afb8586f0d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:52.969607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618eb50-f123-43b4-8bdb-1ee01b3692b8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:53.066253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618eb50-f123-43b4-8bdb-1ee01b3692b8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:53.098606 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe6a0b-96d9-44d6-898e-105979cd8c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:53.099277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe6a0b-96d9-44d6-898e-105979cd8c64 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:53.099622 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe6a0b-96d9-44d6-898e-105979cd8c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:53.099896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe6a0b-96d9-44d6-898e-105979cd8c64 None admin] PATH_INFO: `/v3/projects/a00952d9403b42528c225d5daf466667` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:53.100991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe6a0b-96d9-44d6-898e-105979cd8c64 None admin] RBAC: Authorizing `identity:get_project(project_id=a00952d9403b42528c225d5daf466667)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:53.102402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe6a0b-96d9-44d6-898e-105979cd8c64 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:53.104020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 591/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:03:52 2022] GET /identity/v3/projects/a00952d9403b42528c225d5daf466667 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:53.109790 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86694f53-153d-447e-b642-4a7498aa13d9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:53.207032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86694f53-153d-447e-b642-4a7498aa13d9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:53.240732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce502c0-10a4-45d5-a373-2a3d8ec0d5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:53.241376 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce502c0-10a4-45d5-a373-2a3d8ec0d5d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:53.241799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce502c0-10a4-45d5-a373-2a3d8ec0d5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:53.242155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce502c0-10a4-45d5-a373-2a3d8ec0d5d8 None admin] PATH_INFO: `/v3/projects/a00952d9403b42528c225d5daf466667/users/bd9092501ef64b55b79e57afb8586f0d/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:53.244046 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce502c0-10a4-45d5-a373-2a3d8ec0d5d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a00952d9403b42528c225d5daf466667, user_id=bd9092501ef64b55b79e57afb8586f0d, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:53.245460 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce502c0-10a4-45d5-a373-2a3d8ec0d5d8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:53.247657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:53.258485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 601/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:03:53 2022] PUT /identity/v3/projects/a00952d9403b42528c225d5daf466667/users/bd9092501ef64b55b79e57afb8586f0d/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:55.384003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cfce4-ba76-479a-9f0b-75008e88eb70 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:55.384531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cfce4-ba76-479a-9f0b-75008e88eb70 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:55.384531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cfce4-ba76-479a-9f0b-75008e88eb70 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:55.385856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 592/1193] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:03:55.390990 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288dad3-8452-4df4-9d8f-a95fe4d4b24d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:55.391431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288dad3-8452-4df4-9d8f-a95fe4d4b24d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:55.391543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288dad3-8452-4df4-9d8f-a95fe4d4b24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:55.409728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6288dad3-8452-4df4-9d8f-a95fe4d4b24d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:55.411545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6288dad3-8452-4df4-9d8f-a95fe4d4b24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:55.476377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 602/1194] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:03:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:03:55.747329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cfce4-ba76-479a-9f0b-75008e88eb70 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:55.774921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:55.849467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cfce4-ba76-479a-9f0b-75008e88eb70 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:55.881079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d088ba-2879-4b4d-a94e-e60c10502a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:55.881683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d088ba-2879-4b4d-a94e-e60c10502a1e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:55.881956 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d088ba-2879-4b4d-a94e-e60c10502a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:55.882220 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d088ba-2879-4b4d-a94e-e60c10502a1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:55.892361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d088ba-2879-4b4d-a94e-e60c10502a1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-82d088ba-2879-4b4d-a94e-e60c10502a1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:03:55.897688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:03:55.898814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:03:55.899720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:03:55.899720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:03:55.899720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:03:55.899720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:03:55.899720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:03:55.899720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:03:55.899720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:03:55.899720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:03:55.899720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:03:55.899720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:03:55.899720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:03:55.899720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:03:55.899720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 593/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:03:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:03:55.904675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288dad3-8452-4df4-9d8f-a95fe4d4b24d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:56.007624 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288dad3-8452-4df4-9d8f-a95fe4d4b24d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:56.029894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:56.041019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e15272-8029-4f40-9bb3-298db3eec255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:56.041620 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e15272-8029-4f40-9bb3-298db3eec255 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:56.041902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e15272-8029-4f40-9bb3-298db3eec255 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:56.042173 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e15272-8029-4f40-9bb3-298db3eec255 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:56.042822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e15272-8029-4f40-9bb3-298db3eec255 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:56.043156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e15272-8029-4f40-9bb3-298db3eec255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:56.044510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e15272-8029-4f40-9bb3-298db3eec255 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:56.052835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 603/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:03:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:56.058970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d088ba-2879-4b4d-a94e-e60c10502a1e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:56.141607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:56.157479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d088ba-2879-4b4d-a94e-e60c10502a1e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:56.191061 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a0956-4ab4-4f9d-ad3d-54b9ee3a0ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:56.192220 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a0956-4ab4-4f9d-ad3d-54b9ee3a0ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:56.192655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a0956-4ab4-4f9d-ad3d-54b9ee3a0ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:56.192942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a0956-4ab4-4f9d-ad3d-54b9ee3a0ab8 None admin] PATH_INFO: `/v3/users/bd9092501ef64b55b79e57afb8586f0d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:56.194073 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a0956-4ab4-4f9d-ad3d-54b9ee3a0ab8 None admin] RBAC: Authorizing `identity:get_user(user_id=bd9092501ef64b55b79e57afb8586f0d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:56.195536 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a0956-4ab4-4f9d-ad3d-54b9ee3a0ab8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:56.197103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 594/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 9 13:03:56 2022] GET /identity/v3/users/bd9092501ef64b55b79e57afb8586f0d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:56.202797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e15272-8029-4f40-9bb3-298db3eec255 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:56.299974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e15272-8029-4f40-9bb3-298db3eec255 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:56.333658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7988d-afd9-43e6-9ee9-491d6b779a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:56.334253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7988d-afd9-43e6-9ee9-491d6b779a4c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:56.334530 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7988d-afd9-43e6-9ee9-491d6b779a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:56.334798 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7988d-afd9-43e6-9ee9-491d6b779a4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:56.392720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:56.406513 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc7988d-afd9-43e6-9ee9-491d6b779a4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:56.407996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc7988d-afd9-43e6-9ee9-491d6b779a4c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:56.502737 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 604/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:03:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:03:56.509255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a0956-4ab4-4f9d-ad3d-54b9ee3a0ab8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:56.607038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a0956-4ab4-4f9d-ad3d-54b9ee3a0ab8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:56.640815 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9deca-5d92-4a1d-bb6a-489e788c35f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:56.641488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9deca-5d92-4a1d-bb6a-489e788c35f7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:56.641768 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9deca-5d92-4a1d-bb6a-489e788c35f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:56.642041 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9deca-5d92-4a1d-bb6a-489e788c35f7 None admin] PATH_INFO: `/v3/projects/a00952d9403b42528c225d5daf466667` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:56.643179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9deca-5d92-4a1d-bb6a-489e788c35f7 None admin] RBAC: Authorizing `identity:get_project(project_id=a00952d9403b42528c225d5daf466667)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:56.644631 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9deca-5d92-4a1d-bb6a-489e788c35f7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:56.646286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 595/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 9 13:03:56 2022] GET /identity/v3/projects/a00952d9403b42528c225d5daf466667 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:56.652377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7988d-afd9-43e6-9ee9-491d6b779a4c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:56.733814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:56.748957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7988d-afd9-43e6-9ee9-491d6b779a4c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:56.782673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4532e3-ac37-4d8e-b5d3-36ab1bc89fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:56.783302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4532e3-ac37-4d8e-b5d3-36ab1bc89fce None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:56.783580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4532e3-ac37-4d8e-b5d3-36ab1bc89fce None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:56.783854 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4532e3-ac37-4d8e-b5d3-36ab1bc89fce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:56.784565 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4532e3-ac37-4d8e-b5d3-36ab1bc89fce None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=a00952d9403b42528c225d5daf466667, user.id=bd9092501ef64b55b79e57afb8586f0d) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:56.784868 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4532e3-ac37-4d8e-b5d3-36ab1bc89fce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:56.786507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4532e3-ac37-4d8e-b5d3-36ab1bc89fce None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:56.792709 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 605/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:03:56 2022] GET /identity/v3/role_assignments?user.id=bd9092501ef64b55b79e57afb8586f0d&scope.project.id=a00952d9403b42528c225d5daf466667&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:04:06.636683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:06.790138 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=25 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:52.730661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d618af-edde-4018-87c5-b596fd5f5d79 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:52.731382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d618af-edde-4018-87c5-b596fd5f5d79 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:52.731382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d618af-edde-4018-87c5-b596fd5f5d79 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:52.735797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 596/1201] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:04:52.739203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4350c-8ff5-46ed-8a3a-90cb365db09d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:52.739674 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4350c-8ff5-46ed-8a3a-90cb365db09d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:52.739788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4350c-8ff5-46ed-8a3a-90cb365db09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:52.759321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b2a4350c-8ff5-46ed-8a3a-90cb365db09d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:52.760541 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b2a4350c-8ff5-46ed-8a3a-90cb365db09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:52.842681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 606/1202] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:04:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:04:53.116784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d618af-edde-4018-87c5-b596fd5f5d79 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:53.239364 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d618af-edde-4018-87c5-b596fd5f5d79 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:53.285375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826405a-5726-4e71-9186-28be97466c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:53.286006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826405a-5726-4e71-9186-28be97466c6f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:53.286289 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826405a-5726-4e71-9186-28be97466c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:53.286564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826405a-5726-4e71-9186-28be97466c6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:53.303082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9826405a-5726-4e71-9186-28be97466c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:53.303082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9826405a-5726-4e71-9186-28be97466c6f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9826405a-5726-4e71-9186-28be97466c6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:04:53.317785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:04:53.319179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:04:53.320397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:04:53.320397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:04:53.320397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:04:53.324559 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 597/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:04:53.325619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4350c-8ff5-46ed-8a3a-90cb365db09d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:53.464188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4350c-8ff5-46ed-8a3a-90cb365db09d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:53.506734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:53.513787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b4bd6-8bdc-4bb8-84fc-5a16ae81de6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:53.514568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4bd6-8bdc-4bb8-84fc-5a16ae81de6c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:53.514974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4bd6-8bdc-4bb8-84fc-5a16ae81de6c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:53.515324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4bd6-8bdc-4bb8-84fc-5a16ae81de6c None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:53.516028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b4bd6-8bdc-4bb8-84fc-5a16ae81de6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:04:53.516341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b4bd6-8bdc-4bb8-84fc-5a16ae81de6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:53.518129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b4bd6-8bdc-4bb8-84fc-5a16ae81de6c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:53.531462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 607/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:04:53.537195 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826405a-5726-4e71-9186-28be97466c6f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:53.604349 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:53.677345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826405a-5726-4e71-9186-28be97466c6f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:53.738080 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47e3dc-530b-456c-9379-9eb543fde537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:53.738867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47e3dc-530b-456c-9379-9eb543fde537 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:53.738867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47e3dc-530b-456c-9379-9eb543fde537 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:53.739257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47e3dc-530b-456c-9379-9eb543fde537 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:53.739837 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47e3dc-530b-456c-9379-9eb543fde537 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:53.741322 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47e3dc-530b-456c-9379-9eb543fde537 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:53.767431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 598/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 9 13:04:53 2022] POST /identity/v3/users => generated 251 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:04:56.328140 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845dcc9-57b7-4346-9ad7-69993456ee0e None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:56.328763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845dcc9-57b7-4346-9ad7-69993456ee0e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:56.328763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845dcc9-57b7-4346-9ad7-69993456ee0e None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:56.330223 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 608/1206] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:04:56.337275 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921b21d-831c-47dd-b502-deb251904250 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:56.337883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921b21d-831c-47dd-b502-deb251904250 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:56.337883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921b21d-831c-47dd-b502-deb251904250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:56.359910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7921b21d-831c-47dd-b502-deb251904250 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:56.361261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7921b21d-831c-47dd-b502-deb251904250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:56.443616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 599/1207] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:04:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:04:56.733141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845dcc9-57b7-4346-9ad7-69993456ee0e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:56.864874 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845dcc9-57b7-4346-9ad7-69993456ee0e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:56.910835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a163274-c492-4ba3-bb0a-40d480503688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:56.911551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a163274-c492-4ba3-bb0a-40d480503688 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:56.911991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a163274-c492-4ba3-bb0a-40d480503688 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:56.912417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a163274-c492-4ba3-bb0a-40d480503688 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:56.927917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a163274-c492-4ba3-bb0a-40d480503688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-4a163274-c492-4ba3-bb0a-40d480503688 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:04:56.933454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:04:56.934463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:04:56.939588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:04:56.939588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:04:56.939588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:04:56.939588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:04:56.939588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:04:56.939588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:04:56.939588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:04:56.939588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:04:56.939588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:04:56.939588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:04:56.939588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:04:56.939588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:04:56.939588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 609/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:04:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:04:56.942991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921b21d-831c-47dd-b502-deb251904250 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:56.996793 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:57.085740 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921b21d-831c-47dd-b502-deb251904250 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:57.126609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e30264-f6ab-418c-bb47-bd29431b8911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:57.126971 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e30264-f6ab-418c-bb47-bd29431b8911 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:57.127151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e30264-f6ab-418c-bb47-bd29431b8911 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:57.127381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e30264-f6ab-418c-bb47-bd29431b8911 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:57.128132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e30264-f6ab-418c-bb47-bd29431b8911 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:04:57.128409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e30264-f6ab-418c-bb47-bd29431b8911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:57.129870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e30264-f6ab-418c-bb47-bd29431b8911 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:57.143798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 600/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:04:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:04:57.144965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a163274-c492-4ba3-bb0a-40d480503688 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:57.206788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:57.261898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a163274-c492-4ba3-bb0a-40d480503688 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:57.303842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356003e1-2831-4d1a-8cc4-622248fe1098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:57.304490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356003e1-2831-4d1a-8cc4-622248fe1098 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:57.304800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356003e1-2831-4d1a-8cc4-622248fe1098 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:57.305130 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356003e1-2831-4d1a-8cc4-622248fe1098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:57.318920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356003e1-2831-4d1a-8cc4-622248fe1098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:57.320720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356003e1-2831-4d1a-8cc4-622248fe1098 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-356003e1-2831-4d1a-8cc4-622248fe1098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:04:57.337678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:04:57.340396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:04:57.341312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:04:57.341312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:04:57.341312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:04:57.341312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 610/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:04:57.347942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e30264-f6ab-418c-bb47-bd29431b8911 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:57.474570 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:57.478130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e30264-f6ab-418c-bb47-bd29431b8911 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:57.517224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec54eb-25cb-499a-956b-8eb17940b74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:57.518115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec54eb-25cb-499a-956b-8eb17940b74c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:57.518391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec54eb-25cb-499a-956b-8eb17940b74c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:57.518659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec54eb-25cb-499a-956b-8eb17940b74c None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:57.519434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec54eb-25cb-499a-956b-8eb17940b74c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:04:57.519784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec54eb-25cb-499a-956b-8eb17940b74c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:57.521316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec54eb-25cb-499a-956b-8eb17940b74c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:57.535486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 601/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:04:57.541626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356003e1-2831-4d1a-8cc4-622248fe1098 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:57.656460 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356003e1-2831-4d1a-8cc4-622248fe1098 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:57.658735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:57.699678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de0fd6-2e63-4147-acaf-da40d770f460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:57.700320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de0fd6-2e63-4147-acaf-da40d770f460 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:57.700599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de0fd6-2e63-4147-acaf-da40d770f460 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:57.700873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de0fd6-2e63-4147-acaf-da40d770f460 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:57.709040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de0fd6-2e63-4147-acaf-da40d770f460 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:57.710462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de0fd6-2e63-4147-acaf-da40d770f460 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-65de0fd6-2e63-4147-acaf-da40d770f460 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:04:57.722453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:04:57.723842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:04:57.725469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:04:57.725469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:04:57.725469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:04:57.725469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:04:57.725469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:04:57.725469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:04:57.725469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:04:57.725469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:04:57.725469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 13:04:57.725469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:04:57.727238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 611/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:04:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:04:57.729629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec54eb-25cb-499a-956b-8eb17940b74c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:57.840458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec54eb-25cb-499a-956b-8eb17940b74c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:57.882007 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc13680-005b-47c4-bc0d-d5f5a48b6786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:57.882856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc13680-005b-47c4-bc0d-d5f5a48b6786 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:57.883337 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc13680-005b-47c4-bc0d-d5f5a48b6786 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:57.883781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc13680-005b-47c4-bc0d-d5f5a48b6786 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:57.884667 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc13680-005b-47c4-bc0d-d5f5a48b6786 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:04:57.885129 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc13680-005b-47c4-bc0d-d5f5a48b6786 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:57.886708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc13680-005b-47c4-bc0d-d5f5a48b6786 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:57.907070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 602/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 9 13:04:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:04:57.913794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de0fd6-2e63-4147-acaf-da40d770f460 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:58.045855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de0fd6-2e63-4147-acaf-da40d770f460 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:58.081923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0224c-4d24-40c2-9928-7ae08bb9c81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:58.082540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0224c-4d24-40c2-9928-7ae08bb9c81d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:58.082822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0224c-4d24-40c2-9928-7ae08bb9c81d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:58.083119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0224c-4d24-40c2-9928-7ae08bb9c81d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:58.128752 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:58.177157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0224c-4d24-40c2-9928-7ae08bb9c81d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:58.178794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0224c-4d24-40c2-9928-7ae08bb9c81d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:58.308663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 612/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:04:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:04:58.310625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc13680-005b-47c4-bc0d-d5f5a48b6786 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:58.334692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:58.441479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc13680-005b-47c4-bc0d-d5f5a48b6786 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:58.484943 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a3468-8067-49f2-800a-00d61174970a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:58.485706 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a3468-8067-49f2-800a-00d61174970a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:58.486082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a3468-8067-49f2-800a-00d61174970a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:58.486419 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a3468-8067-49f2-800a-00d61174970a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:58.498309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a3468-8067-49f2-800a-00d61174970a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:58.500312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a3468-8067-49f2-800a-00d61174970a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-265a3468-8067-49f2-800a-00d61174970a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:04:58.517990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:04:58.519328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:04:58.520483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:04:58.520483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:04:58.520483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:04:58.526775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 603/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:04:58.528226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0224c-4d24-40c2-9928-7ae08bb9c81d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:58.605310 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:58.645667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0224c-4d24-40c2-9928-7ae08bb9c81d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:58.681096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2986763-e87a-4a97-8ef9-063657eb03e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:58.681705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2986763-e87a-4a97-8ef9-063657eb03e5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:58.681985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2986763-e87a-4a97-8ef9-063657eb03e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:58.682259 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2986763-e87a-4a97-8ef9-063657eb03e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:58.682895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2986763-e87a-4a97-8ef9-063657eb03e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:04:58.683232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2986763-e87a-4a97-8ef9-063657eb03e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:58.684589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2986763-e87a-4a97-8ef9-063657eb03e5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:58.695766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 613/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:04:58.701677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a3468-8067-49f2-800a-00d61174970a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:58.780680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:58.817659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a3468-8067-49f2-800a-00d61174970a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:58.855058 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de3564-96cb-42a0-aa6d-c07025489ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:58.855812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de3564-96cb-42a0-aa6d-c07025489ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:58.856015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de3564-96cb-42a0-aa6d-c07025489ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:58.856283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de3564-96cb-42a0-aa6d-c07025489ce3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:58.863726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de3564-96cb-42a0-aa6d-c07025489ce3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:58.865368 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de3564-96cb-42a0-aa6d-c07025489ce3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-53de3564-96cb-42a0-aa6d-c07025489ce3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:04:58.878437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:04:58.879458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:04:58.880612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:04:58.880612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:04:58.880612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:04:58.880612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:04:58.880612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:04:58.880612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:04:58.880612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:04:58.880612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:04:58.880612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 604/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:04:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:04:58.885860 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2986763-e87a-4a97-8ef9-063657eb03e5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:58.998696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2986763-e87a-4a97-8ef9-063657eb03e5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:59.024892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:59.037752 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7efd81-505c-4454-80c2-ebebb66bbd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:59.038364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7efd81-505c-4454-80c2-ebebb66bbd06 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:59.038638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7efd81-505c-4454-80c2-ebebb66bbd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:59.038905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7efd81-505c-4454-80c2-ebebb66bbd06 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:59.039621 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7efd81-505c-4454-80c2-ebebb66bbd06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:04:59.039924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7efd81-505c-4454-80c2-ebebb66bbd06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:59.041338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7efd81-505c-4454-80c2-ebebb66bbd06 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:59.055520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 614/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:04:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:04:59.062760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de3564-96cb-42a0-aa6d-c07025489ce3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:59.184661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de3564-96cb-42a0-aa6d-c07025489ce3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:59.222112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:59.229391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8c5a1-d37a-4fa0-a691-5b96a8ce25c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:59.229993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8c5a1-d37a-4fa0-a691-5b96a8ce25c1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:59.230280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8c5a1-d37a-4fa0-a691-5b96a8ce25c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:59.230542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8c5a1-d37a-4fa0-a691-5b96a8ce25c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:59.231305 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8c5a1-d37a-4fa0-a691-5b96a8ce25c1 None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=dbc7cef7b714463697dc1ee507e60be1) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:04:59.231643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8c5a1-d37a-4fa0-a691-5b96a8ce25c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:59.233173 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8c5a1-d37a-4fa0-a691-5b96a8ce25c1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:59.238693 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 605/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:04:59 2022] GET /identity/v3/role_assignments?user.id=dbc7cef7b714463697dc1ee507e60be1&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:05:01.459354 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7054a-8238-4503-a942-23a61fdf29cc None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:01.459922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7054a-8238-4503-a942-23a61fdf29cc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:01.460051 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7054a-8238-4503-a942-23a61fdf29cc None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:01.461538 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 615/1220] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:05:01.467551 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918c960-92ae-4825-a45d-9b75ab48d5b4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:01.468040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918c960-92ae-4825-a45d-9b75ab48d5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:01.468144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918c960-92ae-4825-a45d-9b75ab48d5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:01.488479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c918c960-92ae-4825-a45d-9b75ab48d5b4 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:05:01.489749 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c918c960-92ae-4825-a45d-9b75ab48d5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:05:01.560861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 606/1221] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:05:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:05:01.850278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7054a-8238-4503-a942-23a61fdf29cc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:01.964697 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7054a-8238-4503-a942-23a61fdf29cc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:02.009315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbdbcb-40a6-4754-a3ac-cecab7eab203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:02.010269 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbdbcb-40a6-4754-a3ac-cecab7eab203 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:02.010430 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbdbcb-40a6-4754-a3ac-cecab7eab203 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:02.010647 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbdbcb-40a6-4754-a3ac-cecab7eab203 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:02.023343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbdbcb-40a6-4754-a3ac-cecab7eab203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-29bbdbcb-40a6-4754-a3ac-cecab7eab203 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:05:02.028391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:02.030086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:02.030928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:02.030928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:02.030928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:02.030928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:02.030928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:05:02.030928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:05:02.030928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:05:02.030928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:05:02.030928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:05:02.030928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:05:02.030928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:05:02.030928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:02.030928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 616/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:05:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:05:02.036341 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918c960-92ae-4825-a45d-9b75ab48d5b4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:02.143305 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918c960-92ae-4825-a45d-9b75ab48d5b4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:02.168966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:02.182491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bb84f-a7bc-410b-af28-e7a9457aed09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:02.183159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bb84f-a7bc-410b-af28-e7a9457aed09 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:02.183436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bb84f-a7bc-410b-af28-e7a9457aed09 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:02.183709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bb84f-a7bc-410b-af28-e7a9457aed09 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:02.184382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bb84f-a7bc-410b-af28-e7a9457aed09 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:02.184763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bb84f-a7bc-410b-af28-e7a9457aed09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:02.187173 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bb84f-a7bc-410b-af28-e7a9457aed09 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:02.196549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 607/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:05:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:02.203213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbdbcb-40a6-4754-a3ac-cecab7eab203 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:02.252078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:02.320777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbdbcb-40a6-4754-a3ac-cecab7eab203 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:02.360371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fd8b7-d48e-43d8-aa6a-0cce4b8badd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:02.361099 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fd8b7-d48e-43d8-aa6a-0cce4b8badd8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:02.361435 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fd8b7-d48e-43d8-aa6a-0cce4b8badd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:02.361770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fd8b7-d48e-43d8-aa6a-0cce4b8badd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:02.374984 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fd8b7-d48e-43d8-aa6a-0cce4b8badd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:02.376849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fd8b7-d48e-43d8-aa6a-0cce4b8badd8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-450fd8b7-d48e-43d8-aa6a-0cce4b8badd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:05:02.393638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:02.394739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:05:02.396079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:05:02.396079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:02.396079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:02.396079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 617/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:02.401336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bb84f-a7bc-410b-af28-e7a9457aed09 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:02.511235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bb84f-a7bc-410b-af28-e7a9457aed09 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:02.547484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f60513-9a7c-441b-a0f5-dcb2816a7cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:02.548117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f60513-9a7c-441b-a0f5-dcb2816a7cce None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:02.548384 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f60513-9a7c-441b-a0f5-dcb2816a7cce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:02.548658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f60513-9a7c-441b-a0f5-dcb2816a7cce None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:02.549308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f60513-9a7c-441b-a0f5-dcb2816a7cce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:02.549857 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f60513-9a7c-441b-a0f5-dcb2816a7cce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:02.551248 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f60513-9a7c-441b-a0f5-dcb2816a7cce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:02.563011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 608/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:02.568833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fd8b7-d48e-43d8-aa6a-0cce4b8badd8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:02.670698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:02.679681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fd8b7-d48e-43d8-aa6a-0cce4b8badd8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:02.727083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3d9c4-0fb2-49cd-b5e2-86d5e4abd98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:02.727881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3d9c4-0fb2-49cd-b5e2-86d5e4abd98a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:02.728243 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3d9c4-0fb2-49cd-b5e2-86d5e4abd98a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:02.728518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3d9c4-0fb2-49cd-b5e2-86d5e4abd98a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:02.737534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3d9c4-0fb2-49cd-b5e2-86d5e4abd98a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:02.739036 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3d9c4-0fb2-49cd-b5e2-86d5e4abd98a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d5d3d9c4-0fb2-49cd-b5e2-86d5e4abd98a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:05:02.751734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:02.753027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:02.759261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:02.759261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:02.759261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:05:02.759261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:05:02.759261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:05:02.759261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:05:02.759261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:05:02.759261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:05:02.759261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 13:05:02.759261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:02.759261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 618/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:02.759992 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f60513-9a7c-441b-a0f5-dcb2816a7cce None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:02.812643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:02.869064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f60513-9a7c-441b-a0f5-dcb2816a7cce None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:02.906326 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ac49a-aed6-45a5-b150-feddc554e5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:02.907234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ac49a-aed6-45a5-b150-feddc554e5fc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:02.907514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ac49a-aed6-45a5-b150-feddc554e5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:02.907784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ac49a-aed6-45a5-b150-feddc554e5fc None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:02.908525 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ac49a-aed6-45a5-b150-feddc554e5fc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:02.908846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ac49a-aed6-45a5-b150-feddc554e5fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:02.910409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ac49a-aed6-45a5-b150-feddc554e5fc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:02.933395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 609/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 9 13:05:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:02.940084 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3d9c4-0fb2-49cd-b5e2-86d5e4abd98a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:03.058578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3d9c4-0fb2-49cd-b5e2-86d5e4abd98a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:03.096144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019f3d2-2c9f-4064-896f-4c6cf6e630a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:03.096823 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019f3d2-2c9f-4064-896f-4c6cf6e630a7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:03.097191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019f3d2-2c9f-4064-896f-4c6cf6e630a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:03.097532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019f3d2-2c9f-4064-896f-4c6cf6e630a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:03.109484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019f3d2-2c9f-4064-896f-4c6cf6e630a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:03.110990 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019f3d2-2c9f-4064-896f-4c6cf6e630a7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-b019f3d2-2c9f-4064-896f-4c6cf6e630a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:05:03.127142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:03.128285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:05:03.129198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:05:03.129198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:03.129198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:03.129198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 619/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:03.134245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ac49a-aed6-45a5-b150-feddc554e5fc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:03.243957 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ac49a-aed6-45a5-b150-feddc554e5fc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:03.246631 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:03.282330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4155b-627e-454e-97dd-ba5800e3f1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:03.282941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4155b-627e-454e-97dd-ba5800e3f1b1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:03.283249 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4155b-627e-454e-97dd-ba5800e3f1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:03.283609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4155b-627e-454e-97dd-ba5800e3f1b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:03.284158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4155b-627e-454e-97dd-ba5800e3f1b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:03.284567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4155b-627e-454e-97dd-ba5800e3f1b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:03.286107 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4155b-627e-454e-97dd-ba5800e3f1b1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:03.297787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 610/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:03.304284 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019f3d2-2c9f-4064-896f-4c6cf6e630a7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:03.314733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:03.418258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019f3d2-2c9f-4064-896f-4c6cf6e630a7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:03.460210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052be838-254a-4ae4-b653-e8193fa55540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:03.460984 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052be838-254a-4ae4-b653-e8193fa55540 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:03.461271 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052be838-254a-4ae4-b653-e8193fa55540 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:03.461586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052be838-254a-4ae4-b653-e8193fa55540 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:03.469327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052be838-254a-4ae4-b653-e8193fa55540 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:03.470966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052be838-254a-4ae4-b653-e8193fa55540 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-052be838-254a-4ae4-b653-e8193fa55540 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:05:03.484089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:03.485278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:03.491321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:05:03.491321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:05:03.491321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:05:03.491321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:05:03.491321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:05:03.491321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:05:03.491321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:05:03.491321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:03.491321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 620/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:05:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:05:03.491920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4155b-627e-454e-97dd-ba5800e3f1b1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:03.604918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4155b-627e-454e-97dd-ba5800e3f1b1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:03.641631 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b3fe1-0842-48f3-aeab-3f646a8302fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:03.642242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b3fe1-0842-48f3-aeab-3f646a8302fd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:03.642524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b3fe1-0842-48f3-aeab-3f646a8302fd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:03.642796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b3fe1-0842-48f3-aeab-3f646a8302fd None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:03.643519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b3fe1-0842-48f3-aeab-3f646a8302fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:03.643825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b3fe1-0842-48f3-aeab-3f646a8302fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:03.645253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b3fe1-0842-48f3-aeab-3f646a8302fd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:03.658056 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 611/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:05:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:05:03.664311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052be838-254a-4ae4-b653-e8193fa55540 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:03.736828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:03.778149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052be838-254a-4ae4-b653-e8193fa55540 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:03.817251 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee1e34-ce09-4bf6-b317-253cfe75ef5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:03.817912 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee1e34-ce09-4bf6-b317-253cfe75ef5a None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:03.818194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee1e34-ce09-4bf6-b317-253cfe75ef5a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:03.818467 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee1e34-ce09-4bf6-b317-253cfe75ef5a None admin] PATH_INFO: `/v3/projects/2b842ac074b44e148c15882a7964aa13/users/dbc7cef7b714463697dc1ee507e60be1/roles/6e17076d694e40c29dd5e2dbc6998270` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:03.834467 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee1e34-ce09-4bf6-b317-253cfe75ef5a None admin] RBAC: Authorizing `identity:create_grant(project_id=2b842ac074b44e148c15882a7964aa13, user_id=dbc7cef7b714463697dc1ee507e60be1, role_id=6e17076d694e40c29dd5e2dbc6998270)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:03.835992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee1e34-ce09-4bf6-b317-253cfe75ef5a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:03.849343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 621/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:05:03 2022] PUT /identity/v3/projects/2b842ac074b44e148c15882a7964aa13/users/dbc7cef7b714463697dc1ee507e60be1/roles/6e17076d694e40c29dd5e2dbc6998270 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:05:06.344361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda67872-876e-43a6-bb6e-7dfa7970290a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:06.344972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda67872-876e-43a6-bb6e-7dfa7970290a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:06.345092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda67872-876e-43a6-bb6e-7dfa7970290a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:06.346660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 612/1233] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:05:06.352154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5c2d5-0846-4b7d-9aea-46d08316bbe5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:06.352717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5c2d5-0846-4b7d-9aea-46d08316bbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:06.352717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5c2d5-0846-4b7d-9aea-46d08316bbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:06.372144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dda5c2d5-0846-4b7d-9aea-46d08316bbe5 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:05:06.373455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dda5c2d5-0846-4b7d-9aea-46d08316bbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:05:06.448357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 622/1234] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:05:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 13:05:06.738682 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda67872-876e-43a6-bb6e-7dfa7970290a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:06.768027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:06.860814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda67872-876e-43a6-bb6e-7dfa7970290a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:06.900654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa263b-5144-4ef4-8a63-144327edb012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:06.901342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa263b-5144-4ef4-8a63-144327edb012 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:06.901631 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa263b-5144-4ef4-8a63-144327edb012 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:06.901941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa263b-5144-4ef4-8a63-144327edb012 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:06.913832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa263b-5144-4ef4-8a63-144327edb012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-43fa263b-5144-4ef4-8a63-144327edb012 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:05:06.918767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:06.920677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:06.921794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:06.921794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:06.921794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:06.921794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:06.921794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:05:06.921794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:05:06.921794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:05:06.921794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:05:06.921794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:05:06.921794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:05:06.921794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 13:05:06.921794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:05:06.921794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 613/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 9 13:05:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:06.927214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5c2d5-0846-4b7d-9aea-46d08316bbe5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:06.986520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:07.053167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5c2d5-0846-4b7d-9aea-46d08316bbe5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:07.089893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e1751-2db3-4176-bbf6-860e37f2840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:07.090594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e1751-2db3-4176-bbf6-860e37f2840b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:07.090899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e1751-2db3-4176-bbf6-860e37f2840b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:07.091519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e1751-2db3-4176-bbf6-860e37f2840b None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:07.092308 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e1751-2db3-4176-bbf6-860e37f2840b None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:07.092650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e1751-2db3-4176-bbf6-860e37f2840b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:07.094500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e1751-2db3-4176-bbf6-860e37f2840b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:07.109644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 623/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 9 13:05:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:05:07.111374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa263b-5144-4ef4-8a63-144327edb012 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:07.203978 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:07.223152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa263b-5144-4ef4-8a63-144327edb012 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:07.262096 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68bf77-e43b-42cb-8413-36282ba723ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:07.263245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68bf77-e43b-42cb-8413-36282ba723ee None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:07.263728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68bf77-e43b-42cb-8413-36282ba723ee None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:07.264023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68bf77-e43b-42cb-8413-36282ba723ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:07.273960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68bf77-e43b-42cb-8413-36282ba723ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:07.275424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68bf77-e43b-42cb-8413-36282ba723ee None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-da68bf77-e43b-42cb-8413-36282ba723ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:05:07.290421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:07.291601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:05:07.292574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:05:07.292574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:07.292574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:05:07.292574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 614/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:05:07.297504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e1751-2db3-4176-bbf6-860e37f2840b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:07.406637 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e1751-2db3-4176-bbf6-860e37f2840b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:07.416041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:07.443074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf16f6-9d31-443f-9dc1-fd0303f49025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:07.443722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf16f6-9d31-443f-9dc1-fd0303f49025 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:07.443982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf16f6-9d31-443f-9dc1-fd0303f49025 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:07.444262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf16f6-9d31-443f-9dc1-fd0303f49025 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:07.444908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf16f6-9d31-443f-9dc1-fd0303f49025 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:07.445235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf16f6-9d31-443f-9dc1-fd0303f49025 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:07.446589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf16f6-9d31-443f-9dc1-fd0303f49025 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:07.458728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 624/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:07.464962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68bf77-e43b-42cb-8413-36282ba723ee None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:07.577054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68bf77-e43b-42cb-8413-36282ba723ee None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:07.619110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe6440-eb77-47b3-9e6a-cdd4814f302f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:07.619691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe6440-eb77-47b3-9e6a-cdd4814f302f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:07.619984 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe6440-eb77-47b3-9e6a-cdd4814f302f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:07.620235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe6440-eb77-47b3-9e6a-cdd4814f302f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:07.623680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:07.629979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe6440-eb77-47b3-9e6a-cdd4814f302f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:07.631700 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe6440-eb77-47b3-9e6a-cdd4814f302f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-febe6440-eb77-47b3-9e6a-cdd4814f302f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:05:07.643819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:07.644905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:07.645882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:07.645882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:07.645882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:05:07.645882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:05:07.645882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:05:07.645882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:05:07.645882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:05:07.645882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:05:07.645882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 13:05:07.645882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:05:07.645882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 615/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:07.651004 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf16f6-9d31-443f-9dc1-fd0303f49025 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:07.761314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf16f6-9d31-443f-9dc1-fd0303f49025 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:07.797605 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fef4e-f1ae-4b73-99aa-d33563f69d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:07.798209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fef4e-f1ae-4b73-99aa-d33563f69d91 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:07.798486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fef4e-f1ae-4b73-99aa-d33563f69d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:07.798750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fef4e-f1ae-4b73-99aa-d33563f69d91 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:07.799767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fef4e-f1ae-4b73-99aa-d33563f69d91 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:07.800077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fef4e-f1ae-4b73-99aa-d33563f69d91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:07.801502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fef4e-f1ae-4b73-99aa-d33563f69d91 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:07.826457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 625/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 9 13:05:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:07.828088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe6440-eb77-47b3-9e6a-cdd4814f302f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:07.946785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe6440-eb77-47b3-9e6a-cdd4814f302f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:07.985932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f197a-2e48-4f3c-989b-6c1e6c2f52c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:07.986671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f197a-2e48-4f3c-989b-6c1e6c2f52c3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:07.987026 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f197a-2e48-4f3c-989b-6c1e6c2f52c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:07.987349 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f197a-2e48-4f3c-989b-6c1e6c2f52c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:08.062770 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:08.070494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f197a-2e48-4f3c-989b-6c1e6c2f52c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:08.072027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f197a-2e48-4f3c-989b-6c1e6c2f52c3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:08.179026 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 616/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:05:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:05:08.185909 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fef4e-f1ae-4b73-99aa-d33563f69d91 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:08.217912 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:08.302114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fef4e-f1ae-4b73-99aa-d33563f69d91 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:08.338492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffb979-16a3-4760-82d6-83d569c4b716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:08.339056 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffb979-16a3-4760-82d6-83d569c4b716 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:08.339350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffb979-16a3-4760-82d6-83d569c4b716 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:08.339622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffb979-16a3-4760-82d6-83d569c4b716 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:08.352527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dffb979-16a3-4760-82d6-83d569c4b716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:08.355003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dffb979-16a3-4760-82d6-83d569c4b716 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-1dffb979-16a3-4760-82d6-83d569c4b716 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:05:08.371845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:08.373507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:05:08.374421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:05:08.374421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:08.374421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:08.374421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 626/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:08.381873 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f197a-2e48-4f3c-989b-6c1e6c2f52c3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:08.479966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:08.487199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f197a-2e48-4f3c-989b-6c1e6c2f52c3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:08.529581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4aa1d0-5384-4316-ab7d-99e99955e258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:08.530004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aa1d0-5384-4316-ab7d-99e99955e258 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:08.530296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aa1d0-5384-4316-ab7d-99e99955e258 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:08.530561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aa1d0-5384-4316-ab7d-99e99955e258 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:08.531239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4aa1d0-5384-4316-ab7d-99e99955e258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:08.531643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4aa1d0-5384-4316-ab7d-99e99955e258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:08.532927 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4aa1d0-5384-4316-ab7d-99e99955e258 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:08.545177 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 617/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:08.551416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffb979-16a3-4760-82d6-83d569c4b716 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:08.635693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:08.659804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffb979-16a3-4760-82d6-83d569c4b716 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:08.694589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027f4ad-27d1-419c-a447-843184f3691e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:08.695256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027f4ad-27d1-419c-a447-843184f3691e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:08.695535 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027f4ad-27d1-419c-a447-843184f3691e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:08.695806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027f4ad-27d1-419c-a447-843184f3691e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:08.703966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027f4ad-27d1-419c-a447-843184f3691e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:08.705509 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027f4ad-27d1-419c-a447-843184f3691e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6027f4ad-27d1-419c-a447-843184f3691e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:05:08.719699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:08.720859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:08.722142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:05:08.722142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:05:08.722142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:05:08.722142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:05:08.722142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:05:08.722142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:05:08.722142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:05:08.722142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:08.722142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 627/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:05:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:08.727615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4aa1d0-5384-4316-ab7d-99e99955e258 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:08.837478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4aa1d0-5384-4316-ab7d-99e99955e258 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:08.875208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bfa06-4495-4073-8632-47e73baf649e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:08.875751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bfa06-4495-4073-8632-47e73baf649e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:08.876029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bfa06-4495-4073-8632-47e73baf649e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:08.876300 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bfa06-4495-4073-8632-47e73baf649e None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:08.876987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8bfa06-4495-4073-8632-47e73baf649e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:08.877387 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8bfa06-4495-4073-8632-47e73baf649e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:08.878722 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8bfa06-4495-4073-8632-47e73baf649e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:08.892266 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 618/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:05:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:08.899205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027f4ad-27d1-419c-a447-843184f3691e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:09.028998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027f4ad-27d1-419c-a447-843184f3691e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:09.068028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dcd56-394a-4f92-9169-ab3171870a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:09.068685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dcd56-394a-4f92-9169-ab3171870a9d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:09.069028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dcd56-394a-4f92-9169-ab3171870a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:09.069308 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dcd56-394a-4f92-9169-ab3171870a9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:09.070145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dcd56-394a-4f92-9169-ab3171870a9d None admin] RBAC: Adding query filter params (role.id=6e17076d694e40c29dd5e2dbc6998270, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=dbc7cef7b714463697dc1ee507e60be1) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:09.070523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dcd56-394a-4f92-9169-ab3171870a9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:09.072047 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dcd56-394a-4f92-9169-ab3171870a9d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:09.078338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 628/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:05:08 2022] GET /identity/v3/role_assignments?user.id=dbc7cef7b714463697dc1ee507e60be1&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=6e17076d694e40c29dd5e2dbc6998270 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:11.483358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76227d49-064c-4ffb-8eb7-a32badc721b5 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:11.483912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76227d49-064c-4ffb-8eb7-a32badc721b5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:11.483912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76227d49-064c-4ffb-8eb7-a32badc721b5 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:11.485239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 619/1247] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:05:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:05:11.490671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5222b2-d068-4c40-8093-27343f90cca9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:11.491139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5222b2-d068-4c40-8093-27343f90cca9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:11.491262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5222b2-d068-4c40-8093-27343f90cca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:11.498255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:11.511768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ad5222b2-d068-4c40-8093-27343f90cca9 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:05:11.514000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ad5222b2-d068-4c40-8093-27343f90cca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:05:11.583659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 629/1248] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:05:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:05:11.850084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76227d49-064c-4ffb-8eb7-a32badc721b5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:11.868385 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:11.963416 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76227d49-064c-4ffb-8eb7-a32badc721b5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:12.002401 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9815b72-902e-48a0-b83e-83c8115a1da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:12.003132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9815b72-902e-48a0-b83e-83c8115a1da1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:12.003404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9815b72-902e-48a0-b83e-83c8115a1da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:12.003726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9815b72-902e-48a0-b83e-83c8115a1da1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:12.015918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9815b72-902e-48a0-b83e-83c8115a1da1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d9815b72-902e-48a0-b83e-83c8115a1da1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:05:12.021625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:12.023124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:12.023991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:12.023991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:12.023991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:12.023991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:12.023991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:05:12.023991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:05:12.023991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:05:12.023991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:05:12.023991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:05:12.023991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:05:12.023991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:05:12.023991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:05:12.023991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 620/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:05:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:12.029979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5222b2-d068-4c40-8093-27343f90cca9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:12.134381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5222b2-d068-4c40-8093-27343f90cca9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:12.166305 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:12.172624 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d3d0b-f0f0-49d0-b068-f836095ce897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:12.173248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d3d0b-f0f0-49d0-b068-f836095ce897 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:12.173523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d3d0b-f0f0-49d0-b068-f836095ce897 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:12.173869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d3d0b-f0f0-49d0-b068-f836095ce897 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:12.174793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d3d0b-f0f0-49d0-b068-f836095ce897 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:12.175156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d3d0b-f0f0-49d0-b068-f836095ce897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:12.176516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d3d0b-f0f0-49d0-b068-f836095ce897 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:12.185216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 630/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:05:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:12.191425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9815b72-902e-48a0-b83e-83c8115a1da1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:12.276391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:12.304288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9815b72-902e-48a0-b83e-83c8115a1da1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:12.342872 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198945b9-90d9-46fd-bf9a-139e6c7fc91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:12.343554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198945b9-90d9-46fd-bf9a-139e6c7fc91f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:12.343919 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198945b9-90d9-46fd-bf9a-139e6c7fc91f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:12.344276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198945b9-90d9-46fd-bf9a-139e6c7fc91f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:12.355805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198945b9-90d9-46fd-bf9a-139e6c7fc91f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:12.357650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198945b9-90d9-46fd-bf9a-139e6c7fc91f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-198945b9-90d9-46fd-bf9a-139e6c7fc91f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:05:12.373198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:12.374407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:05:12.375543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:05:12.375543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:12.375543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:05:12.375543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 621/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:12.380294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d3d0b-f0f0-49d0-b068-f836095ce897 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:12.489041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d3d0b-f0f0-49d0-b068-f836095ce897 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:12.523923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c749d-d4f2-4f1d-b9da-818509a2a81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:12.524667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c749d-d4f2-4f1d-b9da-818509a2a81d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:12.524833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c749d-d4f2-4f1d-b9da-818509a2a81d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:12.525074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c749d-d4f2-4f1d-b9da-818509a2a81d None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:12.525717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c749d-d4f2-4f1d-b9da-818509a2a81d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:12.526023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c749d-d4f2-4f1d-b9da-818509a2a81d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:12.527412 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c749d-d4f2-4f1d-b9da-818509a2a81d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:12.541435 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 631/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:12.549482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198945b9-90d9-46fd-bf9a-139e6c7fc91f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:12.662271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198945b9-90d9-46fd-bf9a-139e6c7fc91f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:12.691644 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:12.700301 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8f10b-4ba1-4bdc-b611-0781f3b791ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:12.700957 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8f10b-4ba1-4bdc-b611-0781f3b791ae None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:12.701276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8f10b-4ba1-4bdc-b611-0781f3b791ae None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:12.701554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8f10b-4ba1-4bdc-b611-0781f3b791ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:12.709733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8f10b-4ba1-4bdc-b611-0781f3b791ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:12.711228 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8f10b-4ba1-4bdc-b611-0781f3b791ae None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bed8f10b-4ba1-4bdc-b611-0781f3b791ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:05:12.723786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:12.729940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:12.731805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:12.731805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:12.731805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:05:12.731805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:05:12.731805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:05:12.731805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:05:12.731805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:05:12.731805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:05:12.731805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 13:05:12.731805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:05:12.731805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 622/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:05:12.732798 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c749d-d4f2-4f1d-b9da-818509a2a81d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:12.795240 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:12.843044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c749d-d4f2-4f1d-b9da-818509a2a81d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:12.879388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc2e29-0f03-42ef-9fcc-713a1c7de930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:12.880007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc2e29-0f03-42ef-9fcc-713a1c7de930 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:12.880281 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc2e29-0f03-42ef-9fcc-713a1c7de930 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:12.880550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc2e29-0f03-42ef-9fcc-713a1c7de930 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:12.881292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc2e29-0f03-42ef-9fcc-713a1c7de930 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:12.881599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc2e29-0f03-42ef-9fcc-713a1c7de930 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:12.883087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc2e29-0f03-42ef-9fcc-713a1c7de930 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:12.907100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 632/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 9 13:05:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:05:12.908588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8f10b-4ba1-4bdc-b611-0781f3b791ae None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:13.027885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8f10b-4ba1-4bdc-b611-0781f3b791ae None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:13.067781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca8f13-acfb-4684-98e9-4d731c33d850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:13.068380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca8f13-acfb-4684-98e9-4d731c33d850 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:13.068659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca8f13-acfb-4684-98e9-4d731c33d850 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:13.068932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca8f13-acfb-4684-98e9-4d731c33d850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:13.126009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:13.139636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca8f13-acfb-4684-98e9-4d731c33d850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:13.141112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca8f13-acfb-4684-98e9-4d731c33d850 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:13.245841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 623/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:05:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:05:13.252301 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc2e29-0f03-42ef-9fcc-713a1c7de930 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:13.349893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc2e29-0f03-42ef-9fcc-713a1c7de930 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:13.362651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:13.383463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a1d73-c5c2-48df-9ea3-5c2e57c2a893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:13.384089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a1d73-c5c2-48df-9ea3-5c2e57c2a893 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:13.384366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a1d73-c5c2-48df-9ea3-5c2e57c2a893 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:13.384636 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a1d73-c5c2-48df-9ea3-5c2e57c2a893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:13.395160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a1d73-c5c2-48df-9ea3-5c2e57c2a893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:13.396957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a1d73-c5c2-48df-9ea3-5c2e57c2a893 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-846a1d73-c5c2-48df-9ea3-5c2e57c2a893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:05:13.411311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:13.416366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:05:13.417492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:05:13.417492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:13.417492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:13.417492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 633/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:13.418174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca8f13-acfb-4684-98e9-4d731c33d850 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:13.503752 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:13.515963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca8f13-acfb-4684-98e9-4d731c33d850 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:13.549928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd081363-5b73-4419-bbd8-3c9a9c50afdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:13.550535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd081363-5b73-4419-bbd8-3c9a9c50afdd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:13.550789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd081363-5b73-4419-bbd8-3c9a9c50afdd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:13.551082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd081363-5b73-4419-bbd8-3c9a9c50afdd None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:13.551774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd081363-5b73-4419-bbd8-3c9a9c50afdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:13.552083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd081363-5b73-4419-bbd8-3c9a9c50afdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:13.553468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd081363-5b73-4419-bbd8-3c9a9c50afdd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:13.564469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 624/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:13.570071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a1d73-c5c2-48df-9ea3-5c2e57c2a893 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:13.666782 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a1d73-c5c2-48df-9ea3-5c2e57c2a893 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:13.701284 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9ce06-7af4-4168-aa1c-199501385aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:13.701902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9ce06-7af4-4168-aa1c-199501385aab None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:13.702181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9ce06-7af4-4168-aa1c-199501385aab None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:13.702453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9ce06-7af4-4168-aa1c-199501385aab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:13.708848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9ce06-7af4-4168-aa1c-199501385aab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:13.710279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9ce06-7af4-4168-aa1c-199501385aab None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a6a9ce06-7af4-4168-aa1c-199501385aab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:05:13.721468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:13.722504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:13.723654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:05:13.723654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:05:13.723654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:05:13.723654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:05:13.723654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:05:13.723654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:05:13.723654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:05:13.723654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:13.723654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 634/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:05:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:13.728417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd081363-5b73-4419-bbd8-3c9a9c50afdd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:13.826733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd081363-5b73-4419-bbd8-3c9a9c50afdd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:13.859449 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc60146-6ca1-400c-8ad9-f7a1acb9bb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:13.860049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc60146-6ca1-400c-8ad9-f7a1acb9bb69 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:13.860324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc60146-6ca1-400c-8ad9-f7a1acb9bb69 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:13.860595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc60146-6ca1-400c-8ad9-f7a1acb9bb69 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:13.861327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc60146-6ca1-400c-8ad9-f7a1acb9bb69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:13.861592 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc60146-6ca1-400c-8ad9-f7a1acb9bb69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:13.863018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc60146-6ca1-400c-8ad9-f7a1acb9bb69 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:13.875118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 625/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:05:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:13.881134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9ce06-7af4-4168-aa1c-199501385aab None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:13.906309 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:13.982285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9ce06-7af4-4168-aa1c-199501385aab None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:14.022883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17a403-91b4-4126-8633-f660b0d54670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:14.023557 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17a403-91b4-4126-8633-f660b0d54670 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:14.023925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17a403-91b4-4126-8633-f660b0d54670 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:14.024158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17a403-91b4-4126-8633-f660b0d54670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:14.024922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17a403-91b4-4126-8633-f660b0d54670 None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=dbc7cef7b714463697dc1ee507e60be1) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:14.025230 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17a403-91b4-4126-8633-f660b0d54670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:14.026664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17a403-91b4-4126-8633-f660b0d54670 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:14.032055 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 635/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:05:13 2022] GET /identity/v3/role_assignments?user.id=dbc7cef7b714463697dc1ee507e60be1&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:16.156373 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a881c8f-adc7-4525-a04c-6d61e27ca1e7 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:16.156934 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a881c8f-adc7-4525-a04c-6d61e27ca1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:16.156934 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a881c8f-adc7-4525-a04c-6d61e27ca1e7 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:16.158255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 626/1261] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:05:16.163277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5c017-ae90-4545-9077-344c2992a8cc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:16.163643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5c017-ae90-4545-9077-344c2992a8cc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:16.163837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5c017-ae90-4545-9077-344c2992a8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:16.182627 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-99d5c017-ae90-4545-9077-344c2992a8cc None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:05:16.183933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-99d5c017-ae90-4545-9077-344c2992a8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:05:16.248272 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 636/1262] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:05:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:05:16.514585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a881c8f-adc7-4525-a04c-6d61e27ca1e7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:16.525431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:16.616841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a881c8f-adc7-4525-a04c-6d61e27ca1e7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:16.649885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2b433-b054-4463-aa5d-52374e6937f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:16.650515 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2b433-b054-4463-aa5d-52374e6937f6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:16.650793 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2b433-b054-4463-aa5d-52374e6937f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:16.651083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2b433-b054-4463-aa5d-52374e6937f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:16.661515 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2b433-b054-4463-aa5d-52374e6937f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-06c2b433-b054-4463-aa5d-52374e6937f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:05:16.666271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:16.667398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:16.668802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:16.668802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:16.668802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:16.668802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:16.668802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:05:16.668802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:05:16.668802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:05:16.668802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:05:16.668802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:05:16.668802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:05:16.668802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:05:16.668802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:05:16.668802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 627/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:05:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:16.674464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5c017-ae90-4545-9077-344c2992a8cc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:16.692464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:16.771367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5c017-ae90-4545-9077-344c2992a8cc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:16.805179 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364fe40-9c84-492f-8739-f16c2b3d165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:16.805779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364fe40-9c84-492f-8739-f16c2b3d165f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:16.806057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364fe40-9c84-492f-8739-f16c2b3d165f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:16.806328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364fe40-9c84-492f-8739-f16c2b3d165f None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:16.806986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364fe40-9c84-492f-8739-f16c2b3d165f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:16.807318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364fe40-9c84-492f-8739-f16c2b3d165f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:16.808672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364fe40-9c84-492f-8739-f16c2b3d165f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:16.816603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 637/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:05:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:16.822299 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2b433-b054-4463-aa5d-52374e6937f6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:16.889568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:16.922395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2b433-b054-4463-aa5d-52374e6937f6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:16.956720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cf403-88ba-409a-b979-a50be185a8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:16.957342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cf403-88ba-409a-b979-a50be185a8fc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:16.957621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cf403-88ba-409a-b979-a50be185a8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:16.957893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cf403-88ba-409a-b979-a50be185a8fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:16.969027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cf403-88ba-409a-b979-a50be185a8fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:16.970468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cf403-88ba-409a-b979-a50be185a8fc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9a4cf403-88ba-409a-b979-a50be185a8fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:05:16.988069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:16.989155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:05:16.990100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:05:16.990100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:16.990100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:05:16.990100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 628/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:16.997395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364fe40-9c84-492f-8739-f16c2b3d165f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:17.092439 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:17.110769 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364fe40-9c84-492f-8739-f16c2b3d165f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:17.143643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6bbf2-f7c9-4534-bc14-86d70301a3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:17.144243 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6bbf2-f7c9-4534-bc14-86d70301a3ee None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:17.144522 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6bbf2-f7c9-4534-bc14-86d70301a3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:17.144793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6bbf2-f7c9-4534-bc14-86d70301a3ee None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:17.145426 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6bbf2-f7c9-4534-bc14-86d70301a3ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:17.145725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6bbf2-f7c9-4534-bc14-86d70301a3ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:17.147099 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6bbf2-f7c9-4534-bc14-86d70301a3ee None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:17.159085 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 638/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:17.164750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cf403-88ba-409a-b979-a50be185a8fc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:17.263848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cf403-88ba-409a-b979-a50be185a8fc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:17.282905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:17.297126 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701349e1-ab62-4638-ab4a-cceb5ff282d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:17.297745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701349e1-ab62-4638-ab4a-cceb5ff282d1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:17.298024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701349e1-ab62-4638-ab4a-cceb5ff282d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:17.298296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701349e1-ab62-4638-ab4a-cceb5ff282d1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:17.305799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701349e1-ab62-4638-ab4a-cceb5ff282d1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:17.307268 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701349e1-ab62-4638-ab4a-cceb5ff282d1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-701349e1-ab62-4638-ab4a-cceb5ff282d1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:05:17.318772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:17.319903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:17.321090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:17.321090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:17.321090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:05:17.321090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:05:17.321090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:05:17.321090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:05:17.321090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:05:17.321090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:05:17.321090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 13:05:17.321090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:05:17.321090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 629/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:17.326377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6bbf2-f7c9-4534-bc14-86d70301a3ee None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:17.424025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6bbf2-f7c9-4534-bc14-86d70301a3ee None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:17.456162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7d8e5-4619-4df4-ae54-ec62a07774bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:17.456757 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7d8e5-4619-4df4-ae54-ec62a07774bf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:17.457030 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7d8e5-4619-4df4-ae54-ec62a07774bf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:17.457299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7d8e5-4619-4df4-ae54-ec62a07774bf None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:17.458008 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7d8e5-4619-4df4-ae54-ec62a07774bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:17.458307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7d8e5-4619-4df4-ae54-ec62a07774bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:17.459744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7d8e5-4619-4df4-ae54-ec62a07774bf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:17.468477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:17.476685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 639/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 9 13:05:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:17.483007 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701349e1-ab62-4638-ab4a-cceb5ff282d1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:17.580121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701349e1-ab62-4638-ab4a-cceb5ff282d1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:17.613754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec2156-1a0f-4e18-b541-fa49ccaf2ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:17.614393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec2156-1a0f-4e18-b541-fa49ccaf2ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:17.614670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec2156-1a0f-4e18-b541-fa49ccaf2ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:17.614941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec2156-1a0f-4e18-b541-fa49ccaf2ba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:17.625933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec2156-1a0f-4e18-b541-fa49ccaf2ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:17.627394 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec2156-1a0f-4e18-b541-fa49ccaf2ba1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f6ec2156-1a0f-4e18-b541-fa49ccaf2ba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:05:17.641229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:17.642367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:05:17.643346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:05:17.643346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:17.643346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:05:17.643346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 630/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:17.648032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7d8e5-4619-4df4-ae54-ec62a07774bf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:17.744355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7d8e5-4619-4df4-ae54-ec62a07774bf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:17.776738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c00520-c43b-4e80-86d4-eafd84ac744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:17.777330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c00520-c43b-4e80-86d4-eafd84ac744e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:17.777656 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c00520-c43b-4e80-86d4-eafd84ac744e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:17.777930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c00520-c43b-4e80-86d4-eafd84ac744e None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:17.778561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c00520-c43b-4e80-86d4-eafd84ac744e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:17.778863 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c00520-c43b-4e80-86d4-eafd84ac744e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:17.780242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c00520-c43b-4e80-86d4-eafd84ac744e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:17.790835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 640/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:17.796857 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec2156-1a0f-4e18-b541-fa49ccaf2ba1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:17.828522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:17.895064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec2156-1a0f-4e18-b541-fa49ccaf2ba1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:17.928100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c76d78-cc3e-4b2f-9adf-3f0663c65cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:17.928713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c76d78-cc3e-4b2f-9adf-3f0663c65cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:17.928988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c76d78-cc3e-4b2f-9adf-3f0663c65cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:17.929257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c76d78-cc3e-4b2f-9adf-3f0663c65cd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:17.936236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c76d78-cc3e-4b2f-9adf-3f0663c65cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:17.937659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c76d78-cc3e-4b2f-9adf-3f0663c65cd0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-31c76d78-cc3e-4b2f-9adf-3f0663c65cd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:05:17.948646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:17.949769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:17.951106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:05:17.951106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:05:17.951106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:05:17.951106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:05:17.951106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:05:17.951106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:05:17.951106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:05:17.951106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:05:17.951106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 631/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:05:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:17.955645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c00520-c43b-4e80-86d4-eafd84ac744e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:18.017046 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:18.059161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c00520-c43b-4e80-86d4-eafd84ac744e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:18.091814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12d257-b50b-43df-8e75-8f7de54971f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:18.092416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12d257-b50b-43df-8e75-8f7de54971f7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:18.092693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12d257-b50b-43df-8e75-8f7de54971f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:18.092964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12d257-b50b-43df-8e75-8f7de54971f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:18.093648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12d257-b50b-43df-8e75-8f7de54971f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:18.093949 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12d257-b50b-43df-8e75-8f7de54971f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:18.095395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12d257-b50b-43df-8e75-8f7de54971f7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:18.107453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 641/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:05:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:18.113557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c76d78-cc3e-4b2f-9adf-3f0663c65cd0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:18.204903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:18.213766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c76d78-cc3e-4b2f-9adf-3f0663c65cd0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:18.246081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8ce90-83de-46af-9e99-73fa42ddbdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:18.247519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8ce90-83de-46af-9e99-73fa42ddbdf7 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:18.247852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8ce90-83de-46af-9e99-73fa42ddbdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:18.248128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8ce90-83de-46af-9e99-73fa42ddbdf7 None admin] PATH_INFO: `/v3/projects/2b842ac074b44e148c15882a7964aa13/users/dbc7cef7b714463697dc1ee507e60be1/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:18.249983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8ce90-83de-46af-9e99-73fa42ddbdf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b842ac074b44e148c15882a7964aa13, user_id=dbc7cef7b714463697dc1ee507e60be1, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:18.251453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8ce90-83de-46af-9e99-73fa42ddbdf7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:18.264250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 632/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 9 13:05:18 2022] PUT /identity/v3/projects/2b842ac074b44e148c15882a7964aa13/users/dbc7cef7b714463697dc1ee507e60be1/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:05:20.614191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768ea8e-4a31-4c46-a4a6-1ef3049a0e8a None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:20.614601 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768ea8e-4a31-4c46-a4a6-1ef3049a0e8a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:20.614733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768ea8e-4a31-4c46-a4a6-1ef3049a0e8a None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:20.616122 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 642/1274] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:05:20.621990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779adf2-93e9-4a80-b16c-92201bc1ae5c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:20.622370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779adf2-93e9-4a80-b16c-92201bc1ae5c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:20.622570 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779adf2-93e9-4a80-b16c-92201bc1ae5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:20.640365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3779adf2-93e9-4a80-b16c-92201bc1ae5c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:05:20.641531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3779adf2-93e9-4a80-b16c-92201bc1ae5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:05:20.710036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 633/1275] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:05:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:05:20.986331 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768ea8e-4a31-4c46-a4a6-1ef3049a0e8a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:21.096414 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768ea8e-4a31-4c46-a4a6-1ef3049a0e8a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:21.113114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:21.130123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eef490-e833-4d8a-bf13-074a691a7b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:21.130745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eef490-e833-4d8a-bf13-074a691a7b09 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:21.131023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eef490-e833-4d8a-bf13-074a691a7b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:21.131319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eef490-e833-4d8a-bf13-074a691a7b09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:21.141679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eef490-e833-4d8a-bf13-074a691a7b09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-43eef490-e833-4d8a-bf13-074a691a7b09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 13:05:21.146526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:21.147663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:21.148838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:21.148838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:21.148838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:21.148838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:21.148838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 13:05:21.148838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 13:05:21.148838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 13:05:21.148838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 13:05:21.148838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 13:05:21.148838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 13:05:21.148838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 13:05:21.148838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:21.148838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 643/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 9 13:05:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 13:05:21.153676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779adf2-93e9-4a80-b16c-92201bc1ae5c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:21.231923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:21.250063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779adf2-93e9-4a80-b16c-92201bc1ae5c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:21.282776 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943ca32-e3d7-4eb2-b6d6-9f91d3edda79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:21.283398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943ca32-e3d7-4eb2-b6d6-9f91d3edda79 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:21.283677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943ca32-e3d7-4eb2-b6d6-9f91d3edda79 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:21.283951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943ca32-e3d7-4eb2-b6d6-9f91d3edda79 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:21.284607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9943ca32-e3d7-4eb2-b6d6-9f91d3edda79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:21.284913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9943ca32-e3d7-4eb2-b6d6-9f91d3edda79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:21.286273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9943ca32-e3d7-4eb2-b6d6-9f91d3edda79 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:21.294006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 634/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 9 13:05:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:21.300261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eef490-e833-4d8a-bf13-074a691a7b09 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:21.398085 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eef490-e833-4d8a-bf13-074a691a7b09 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:21.431544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a99a5-757f-4280-92cb-d25eac30fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:21.432151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a99a5-757f-4280-92cb-d25eac30fa1d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:21.432423 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a99a5-757f-4280-92cb-d25eac30fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:21.432690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a99a5-757f-4280-92cb-d25eac30fa1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:21.443116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a99a5-757f-4280-92cb-d25eac30fa1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:21.444552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a99a5-757f-4280-92cb-d25eac30fa1d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-bb0a99a5-757f-4280-92cb-d25eac30fa1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:05:21.458684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:21.459729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:05:21.460753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:05:21.460753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:21.460753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:21.460753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 644/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:21.466944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943ca32-e3d7-4eb2-b6d6-9f91d3edda79 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:21.567872 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943ca32-e3d7-4eb2-b6d6-9f91d3edda79 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:21.600279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dbd107-3b78-438f-b786-23f21c98ec9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:21.600881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbd107-3b78-438f-b786-23f21c98ec9c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:21.601158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbd107-3b78-438f-b786-23f21c98ec9c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:21.601433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbd107-3b78-438f-b786-23f21c98ec9c None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:21.602067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dbd107-3b78-438f-b786-23f21c98ec9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:21.602372 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dbd107-3b78-438f-b786-23f21c98ec9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:21.603763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dbd107-3b78-438f-b786-23f21c98ec9c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:21.615830 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 635/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:21.622107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a99a5-757f-4280-92cb-d25eac30fa1d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:21.652572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:21.721891 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a99a5-757f-4280-92cb-d25eac30fa1d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:21.754595 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647a7ca-1691-401e-9dd3-e92d08372239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:21.755247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647a7ca-1691-401e-9dd3-e92d08372239 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:21.755527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647a7ca-1691-401e-9dd3-e92d08372239 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:21.755800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647a7ca-1691-401e-9dd3-e92d08372239 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:21.763238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647a7ca-1691-401e-9dd3-e92d08372239 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:21.764664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647a7ca-1691-401e-9dd3-e92d08372239 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-4647a7ca-1691-401e-9dd3-e92d08372239 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 13:05:21.775417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:21.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:21.778178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:21.778178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:21.778178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 13:05:21.778178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 13:05:21.778178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 13:05:21.778178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 13:05:21.778178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 13:05:21.778178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 13:05:21.778178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 13:05:21.778178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:21.778178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 645/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:21.782763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dbd107-3b78-438f-b786-23f21c98ec9c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:21.786491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:21.880710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dbd107-3b78-438f-b786-23f21c98ec9c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:21.913688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8d67e-dade-4d13-9d5e-2832221eb1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:21.914288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8d67e-dade-4d13-9d5e-2832221eb1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:21.914568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8d67e-dade-4d13-9d5e-2832221eb1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:21.914842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8d67e-dade-4d13-9d5e-2832221eb1c7 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:21.915592 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8d67e-dade-4d13-9d5e-2832221eb1c7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:21.915898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8d67e-dade-4d13-9d5e-2832221eb1c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:21.917319 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8d67e-dade-4d13-9d5e-2832221eb1c7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:21.934450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 636/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 9 13:05:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:21.940329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647a7ca-1691-401e-9dd3-e92d08372239 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:22.034498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:22.044919 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647a7ca-1691-401e-9dd3-e92d08372239 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:22.077948 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d22a65-42b0-42c5-9ca8-48957da43f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:22.078548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d22a65-42b0-42c5-9ca8-48957da43f6f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:22.078824 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d22a65-42b0-42c5-9ca8-48957da43f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:22.079118 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d22a65-42b0-42c5-9ca8-48957da43f6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:22.143584 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d22a65-42b0-42c5-9ca8-48957da43f6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:22.144950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d22a65-42b0-42c5-9ca8-48957da43f6f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:22.231388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:22.241605 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 646/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 9 13:05:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 13:05:22.247546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8d67e-dade-4d13-9d5e-2832221eb1c7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:22.308762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:22.346814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8d67e-dade-4d13-9d5e-2832221eb1c7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:22.379342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885c537-d2f2-42bd-9ad9-888f114dd2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:22.379955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885c537-d2f2-42bd-9ad9-888f114dd2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:22.380234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885c537-d2f2-42bd-9ad9-888f114dd2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:22.380500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885c537-d2f2-42bd-9ad9-888f114dd2b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:22.391144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885c537-d2f2-42bd-9ad9-888f114dd2b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:22.392569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885c537-d2f2-42bd-9ad9-888f114dd2b4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c885c537-d2f2-42bd-9ad9-888f114dd2b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 13:05:22.407649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:22.408859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 13:05:22.410141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 13:05:22.410141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 13:05:22.410141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:05:22.410141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 637/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:22.414340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d22a65-42b0-42c5-9ca8-48957da43f6f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:22.509839 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d22a65-42b0-42c5-9ca8-48957da43f6f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:22.542602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e02cc-db47-47ce-87fc-0b9ed92e4137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:22.543506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e02cc-db47-47ce-87fc-0b9ed92e4137 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:22.543845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e02cc-db47-47ce-87fc-0b9ed92e4137 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:22.544165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e02cc-db47-47ce-87fc-0b9ed92e4137 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:22.544848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e02cc-db47-47ce-87fc-0b9ed92e4137 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:22.545151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e02cc-db47-47ce-87fc-0b9ed92e4137 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:22.546508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e02cc-db47-47ce-87fc-0b9ed92e4137 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:22.557589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 647/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 9 13:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:22.563718 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885c537-d2f2-42bd-9ad9-888f114dd2b4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:22.662849 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885c537-d2f2-42bd-9ad9-888f114dd2b4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:22.676844 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:22.696660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5c762-5e28-47a4-a292-ea1fa6fbe810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:22.697276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5c762-5e28-47a4-a292-ea1fa6fbe810 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:22.697549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5c762-5e28-47a4-a292-ea1fa6fbe810 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:22.697816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5c762-5e28-47a4-a292-ea1fa6fbe810 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:22.704285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5c762-5e28-47a4-a292-ea1fa6fbe810 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:22.705707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5c762-5e28-47a4-a292-ea1fa6fbe810 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d3d5c762-5e28-47a4-a292-ea1fa6fbe810 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 13:05:22.717292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:22.718342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:22.719504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 13:05:22.719504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 13:05:22.719504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 13:05:22.719504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 13:05:22.719504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 13:05:22.719504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 13:05:22.719504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 13:05:22.719504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Sep 09 13:05:22.719504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 638/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 9 13:05:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:22.724192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e02cc-db47-47ce-87fc-0b9ed92e4137 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:22.772989 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:22.822226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e02cc-db47-47ce-87fc-0b9ed92e4137 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:22.855894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d5979-527d-41c6-9bd8-07afb40f402c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:22.856486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d5979-527d-41c6-9bd8-07afb40f402c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:22.856766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d5979-527d-41c6-9bd8-07afb40f402c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:22.857035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d5979-527d-41c6-9bd8-07afb40f402c None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:22.857713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d5979-527d-41c6-9bd8-07afb40f402c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:22.858015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d5979-527d-41c6-9bd8-07afb40f402c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:22.859456 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d5979-527d-41c6-9bd8-07afb40f402c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:22.870952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 648/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 9 13:05:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:22.877074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5c762-5e28-47a4-a292-ea1fa6fbe810 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:22.975133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5c762-5e28-47a4-a292-ea1fa6fbe810 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:23.007953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4347bf-d0bb-4d8c-ab8d-202353289f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:23.008555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4347bf-d0bb-4d8c-ab8d-202353289f5c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:23.008832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4347bf-d0bb-4d8c-ab8d-202353289f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:23.009103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4347bf-d0bb-4d8c-ab8d-202353289f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:23.009822 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4347bf-d0bb-4d8c-ab8d-202353289f5c None admin] RBAC: Adding query filter params (role.id=8e1f1c5e55d24237b79880bb43f41ee6, scope.project.id=2b842ac074b44e148c15882a7964aa13, user.id=dbc7cef7b714463697dc1ee507e60be1) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:23.010131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4347bf-d0bb-4d8c-ab8d-202353289f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:23.012182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4347bf-d0bb-4d8c-ab8d-202353289f5c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:23.017806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 639/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 9 13:05:22 2022] GET /identity/v3/role_assignments?user.id=dbc7cef7b714463697dc1ee507e60be1&scope.project.id=2b842ac074b44e148c15882a7964aa13&role.id=8e1f1c5e55d24237b79880bb43f41ee6 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:25.184362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145ecb2-ee4b-403d-b749-6c5742c18b7d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:25.184895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145ecb2-ee4b-403d-b749-6c5742c18b7d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:25.184895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145ecb2-ee4b-403d-b749-6c5742c18b7d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:25.186240 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 649/1288] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:05:25.191518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be25a5-1050-490b-b441-d0bb4156cad2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:25.191906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be25a5-1050-490b-b441-d0bb4156cad2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:25.192075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be25a5-1050-490b-b441-d0bb4156cad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:25.210505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-30be25a5-1050-490b-b441-d0bb4156cad2 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:05:25.211723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-30be25a5-1050-490b-b441-d0bb4156cad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:05:25.240548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:25.277900 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 640/1289] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:05:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:05:25.546425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145ecb2-ee4b-403d-b749-6c5742c18b7d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:25.645371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145ecb2-ee4b-403d-b749-6c5742c18b7d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:25.655744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:25.678929 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d7846-e55d-4d8d-a329-d335ea2610a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:25.679573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d7846-e55d-4d8d-a329-d335ea2610a2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:25.679851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d7846-e55d-4d8d-a329-d335ea2610a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:25.680119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d7846-e55d-4d8d-a329-d335ea2610a2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:25.680690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d7846-e55d-4d8d-a329-d335ea2610a2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:25.682050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d7846-e55d-4d8d-a329-d335ea2610a2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-b22d7846-e55d-4d8d-a329-d335ea2610a2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:25.691039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:25.692393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:25.693596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 650/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 13:05:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:25.698103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be25a5-1050-490b-b441-d0bb4156cad2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:25.795383 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be25a5-1050-490b-b441-d0bb4156cad2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:25.828598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97892189-0d52-4890-887f-dd2743216501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:25.829133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97892189-0d52-4890-887f-dd2743216501 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:25.829410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97892189-0d52-4890-887f-dd2743216501 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:25.829682 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97892189-0d52-4890-887f-dd2743216501 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:25.830309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97892189-0d52-4890-887f-dd2743216501 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:25.830612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97892189-0d52-4890-887f-dd2743216501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:25.832003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97892189-0d52-4890-887f-dd2743216501 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:25.836716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 641/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Sep 9 13:05:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:25.843038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d7846-e55d-4d8d-a329-d335ea2610a2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:25.940906 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d7846-e55d-4d8d-a329-d335ea2610a2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:25.973768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748675e0-31f0-4426-8fd3-7225545b18bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:25.974443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748675e0-31f0-4426-8fd3-7225545b18bf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:25.974773 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748675e0-31f0-4426-8fd3-7225545b18bf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:25.975112 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748675e0-31f0-4426-8fd3-7225545b18bf None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:25.975913 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748675e0-31f0-4426-8fd3-7225545b18bf None admin] RBAC: Adding query filter params (type=placement) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:25.976353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748675e0-31f0-4426-8fd3-7225545b18bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:25.978524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748675e0-31f0-4426-8fd3-7225545b18bf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:25.984606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 651/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Sep 9 13:05:25 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:28.107456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e4dd4-61c1-42c8-a2f4-4cb0785afa1e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:28.108008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e4dd4-61c1-42c8-a2f4-4cb0785afa1e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:28.108008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e4dd4-61c1-42c8-a2f4-4cb0785afa1e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:28.109351 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 642/1293] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:05:28.114553 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454557c-359a-4f81-ac4f-2ceab4ad1165 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:28.114968 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454557c-359a-4f81-ac4f-2ceab4ad1165 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:28.115135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454557c-359a-4f81-ac4f-2ceab4ad1165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:28.133395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f454557c-359a-4f81-ac4f-2ceab4ad1165 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:05:28.134581 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f454557c-359a-4f81-ac4f-2ceab4ad1165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:05:28.163064 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:28.200170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 652/1294] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:05:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:05:28.467267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e4dd4-61c1-42c8-a2f4-4cb0785afa1e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:28.497907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:28.568614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e4dd4-61c1-42c8-a2f4-4cb0785afa1e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:28.601704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d02e8-2c95-4dea-979b-9d1698dcebaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:28.602315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d02e8-2c95-4dea-979b-9d1698dcebaf None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:28.602594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d02e8-2c95-4dea-979b-9d1698dcebaf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:28.602871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d02e8-2c95-4dea-979b-9d1698dcebaf None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:28.603558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d02e8-2c95-4dea-979b-9d1698dcebaf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:28.605479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d02e8-2c95-4dea-979b-9d1698dcebaf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:28.619675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 643/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 9 13:05:28 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:05:30.760332 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12ede3-024c-476b-a5af-5df49c380a96 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:30.760929 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12ede3-024c-476b-a5af-5df49c380a96 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:30.761025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12ede3-024c-476b-a5af-5df49c380a96 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:30.762526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 653/1296] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:05:30.767660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2ad5-f97d-4085-bb0d-747f6aa75615 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:30.768030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2ad5-f97d-4085-bb0d-747f6aa75615 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:30.768203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2ad5-f97d-4085-bb0d-747f6aa75615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:30.786442 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1afa2ad5-f97d-4085-bb0d-747f6aa75615 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:05:30.787648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1afa2ad5-f97d-4085-bb0d-747f6aa75615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:05:30.858696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 644/1297] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:05:30 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:05:31.127520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12ede3-024c-476b-a5af-5df49c380a96 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:31.234159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12ede3-024c-476b-a5af-5df49c380a96 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:31.267964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce54603-a9f0-45e2-aec0-1e12d6c9ee8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:31.268582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce54603-a9f0-45e2-aec0-1e12d6c9ee8b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:31.268856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce54603-a9f0-45e2-aec0-1e12d6c9ee8b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:31.269124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce54603-a9f0-45e2-aec0-1e12d6c9ee8b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:31.269693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce54603-a9f0-45e2-aec0-1e12d6c9ee8b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:31.271035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce54603-a9f0-45e2-aec0-1e12d6c9ee8b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-3ce54603-a9f0-45e2-aec0-1e12d6c9ee8b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:31.280477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:31.281638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:31.282506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 654/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 13:05:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:31.287619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa2ad5-f97d-4085-bb0d-747f6aa75615 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:31.306180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:31.386765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa2ad5-f97d-4085-bb0d-747f6aa75615 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:31.420172 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedbe9b-1700-4d0c-88b8-64d0ca11d4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:31.420763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedbe9b-1700-4d0c-88b8-64d0ca11d4ce None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:31.421045 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedbe9b-1700-4d0c-88b8-64d0ca11d4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:31.421318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedbe9b-1700-4d0c-88b8-64d0ca11d4ce None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:31.421941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedbe9b-1700-4d0c-88b8-64d0ca11d4ce None admin] RBAC: Adding query filter params (name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:31.422245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedbe9b-1700-4d0c-88b8-64d0ca11d4ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:31.423900 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedbe9b-1700-4d0c-88b8-64d0ca11d4ce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:31.428866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 645/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Sep 9 13:05:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:31.435338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce54603-a9f0-45e2-aec0-1e12d6c9ee8b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:31.466205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:31.532753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce54603-a9f0-45e2-aec0-1e12d6c9ee8b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:31.567906 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcac7b8-f913-4cbd-b676-be24c51ead9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:31.568500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcac7b8-f913-4cbd-b676-be24c51ead9f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:31.568776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcac7b8-f913-4cbd-b676-be24c51ead9f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:31.569045 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcac7b8-f913-4cbd-b676-be24c51ead9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:31.569735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcac7b8-f913-4cbd-b676-be24c51ead9f None admin] RBAC: Adding query filter params (interface=public, service_id=b653cdbcc2194b21a6b694ced6fbecba, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:31.570038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcac7b8-f913-4cbd-b676-be24c51ead9f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:31.571413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcac7b8-f913-4cbd-b676-be24c51ead9f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:31.576444 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 655/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 9 13:05:31 2022] GET /identity/v3/endpoints?service_id=b653cdbcc2194b21a6b694ced6fbecba&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:31.582078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedbe9b-1700-4d0c-88b8-64d0ca11d4ce None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:31.662347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:31.681374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedbe9b-1700-4d0c-88b8-64d0ca11d4ce None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:31.715147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bed91c-eef6-4a88-be72-6c148c062639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:31.715745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bed91c-eef6-4a88-be72-6c148c062639 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:31.716023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bed91c-eef6-4a88-be72-6c148c062639 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:31.716290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bed91c-eef6-4a88-be72-6c148c062639 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:31.716879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bed91c-eef6-4a88-be72-6c148c062639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:31.718241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bed91c-eef6-4a88-be72-6c148c062639 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:31.724843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 646/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 9 13:05:31 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:05:33.859850 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9055e-d660-44b9-a0e7-1bf624bc63f1 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:33.860364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9055e-d660-44b9-a0e7-1bf624bc63f1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:33.860364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9055e-d660-44b9-a0e7-1bf624bc63f1 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:33.861650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 656/1302] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:05:33.866869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036d9c5-263b-45ea-8a93-d94091a15152 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:33.867317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036d9c5-263b-45ea-8a93-d94091a15152 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:33.867469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036d9c5-263b-45ea-8a93-d94091a15152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:33.885313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7036d9c5-263b-45ea-8a93-d94091a15152 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:05:33.886568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7036d9c5-263b-45ea-8a93-d94091a15152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:05:33.952535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 647/1303] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:05:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 13:05:34.224732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9055e-d660-44b9-a0e7-1bf624bc63f1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:34.319452 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:34.327327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9055e-d660-44b9-a0e7-1bf624bc63f1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:34.360164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1f6d0-60a9-447d-a5b1-1b9e4ee6e8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:34.360777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1f6d0-60a9-447d-a5b1-1b9e4ee6e8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:34.361051 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1f6d0-60a9-447d-a5b1-1b9e4ee6e8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:34.361320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1f6d0-60a9-447d-a5b1-1b9e4ee6e8d8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:34.361890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1f6d0-60a9-447d-a5b1-1b9e4ee6e8d8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:34.363295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1f6d0-60a9-447d-a5b1-1b9e4ee6e8d8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d1c1f6d0-60a9-447d-a5b1-1b9e4ee6e8d8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 13:05:34.372040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 13:05:34.373338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Sep 09 13:05:34.374490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 657/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 9 13:05:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 13:05:34.379003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036d9c5-263b-45ea-8a93-d94091a15152 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:34.454681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:34.476989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036d9c5-263b-45ea-8a93-d94091a15152 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:34.509783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5f73b-118e-400e-9094-c9127fb7fda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:34.510447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5f73b-118e-400e-9094-c9127fb7fda9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:34.511255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5f73b-118e-400e-9094-c9127fb7fda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:34.511538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5f73b-118e-400e-9094-c9127fb7fda9 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:34.512132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5f73b-118e-400e-9094-c9127fb7fda9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:05:34.512429 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5f73b-118e-400e-9094-c9127fb7fda9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:34.513784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5f73b-118e-400e-9094-c9127fb7fda9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:34.518785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 648/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Sep 9 13:05:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:05:34.524544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1f6d0-60a9-447d-a5b1-1b9e4ee6e8d8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:34.624114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1f6d0-60a9-447d-a5b1-1b9e4ee6e8d8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:34.657585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aea818-4959-4400-b623-f1d84edcf6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:34.658182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea818-4959-4400-b623-f1d84edcf6df None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:34.658465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea818-4959-4400-b623-f1d84edcf6df None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:34.658741 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea818-4959-4400-b623-f1d84edcf6df None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:34.659430 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aea818-4959-4400-b623-f1d84edcf6df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:34.660786 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aea818-4959-4400-b623-f1d84edcf6df None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:34.679903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 658/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:05:34 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:05:44.515945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=30 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:44.675788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:38.632512 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd83fe-6e52-4a95-931f-c7fd1ee0ee69 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:38.632955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd83fe-6e52-4a95-931f-c7fd1ee0ee69 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:38.633198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd83fe-6e52-4a95-931f-c7fd1ee0ee69 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:38.634681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 649/1307] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:06:38.641155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e8348-12ba-4b5f-ac5f-028a3d9d6711 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:38.641578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e8348-12ba-4b5f-ac5f-028a3d9d6711 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:38.641942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e8348-12ba-4b5f-ac5f-028a3d9d6711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:38.781258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b52e8348-12ba-4b5f-ac5f-028a3d9d6711 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:06:38.783494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b52e8348-12ba-4b5f-ac5f-028a3d9d6711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:38.948814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 659/1308] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:06:39.247910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c75260-6976-450b-9397-37d2f563a684 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:39.252702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c75260-6976-450b-9397-37d2f563a684 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:39.252986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c75260-6976-450b-9397-37d2f563a684 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:39.254519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 650/1309] 10.222.0.17 () {54 vars in 965 bytes} [Fri Sep 9 13:06:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:06:39.261609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188178a-77b5-495d-a177-fd9bc748d515 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:39.261994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188178a-77b5-495d-a177-fd9bc748d515 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:39.262343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188178a-77b5-495d-a177-fd9bc748d515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:39.355751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0188178a-77b5-495d-a177-fd9bc748d515 None None] MFA Rules not processed for user `e12deabe077b4095bfa7e2e8fac61d14`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:06:39.357164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0188178a-77b5-495d-a177-fd9bc748d515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:39.381970 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:39.444176 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 660/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Fri Sep 9 13:06:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:06:39.453282 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c75260-6976-450b-9397-37d2f563a684 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:06:39.455249 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-27c75260-6976-450b-9397-37d2f563a684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:39.568081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c75260-6976-450b-9397-37d2f563a684 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:06:39.649445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c2c89-3e36-49e3-81fb-b262ee5b205d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:06:39.650090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c2c89-3e36-49e3-81fb-b262ee5b205d service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:39.650393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c2c89-3e36-49e3-81fb-b262ee5b205d service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:39.650590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c2c89-3e36-49e3-81fb-b262ee5b205d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:39.652220 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c37c2c89-3e36-49e3-81fb-b262ee5b205d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:39.713078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:39.779761 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c2c89-3e36-49e3-81fb-b262ee5b205d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:06:39.781392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c2c89-3e36-49e3-81fb-b262ee5b205d service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:06:39.817912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 651/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:06:42.340398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2f079-bca2-4714-a593-2c9a53d25521 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:42.340958 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2f079-bca2-4714-a593-2c9a53d25521 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:42.340958 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2f079-bca2-4714-a593-2c9a53d25521 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:42.342306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 661/1312] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:06:42.347456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad0355-9e6a-4119-84aa-22f9cb54e8f2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:42.347876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad0355-9e6a-4119-84aa-22f9cb54e8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:42.347988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad0355-9e6a-4119-84aa-22f9cb54e8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:42.417923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a0ad0355-9e6a-4119-84aa-22f9cb54e8f2 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:06:42.419091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a0ad0355-9e6a-4119-84aa-22f9cb54e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:42.513911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 652/1313] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:06:42.798071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2f079-bca2-4714-a593-2c9a53d25521 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:06:42.799679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4ce2f079-bca2-4714-a593-2c9a53d25521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:42.904573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2f079-bca2-4714-a593-2c9a53d25521 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:06:42.976409 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:42.978805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1adec-e213-42f5-9180-7cfa4eac6bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:06:42.979697 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1adec-e213-42f5-9180-7cfa4eac6bfb service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:42.980034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1adec-e213-42f5-9180-7cfa4eac6bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:42.980451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1adec-e213-42f5-9180-7cfa4eac6bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:42.982277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-13e1adec-e213-42f5-9180-7cfa4eac6bfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:43.090722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1adec-e213-42f5-9180-7cfa4eac6bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:06:43.092189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1adec-e213-42f5-9180-7cfa4eac6bfb service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:06:43.110500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 662/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:06:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:06:45.599295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766085a1-4916-4d4b-b58c-f18f9e8c5b1d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:45.599843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766085a1-4916-4d4b-b58c-f18f9e8c5b1d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:45.599843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766085a1-4916-4d4b-b58c-f18f9e8c5b1d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:45.601207 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 653/1315] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:06:45.606512 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3666de-57df-4069-be58-df81bd51c204 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:45.606942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3666de-57df-4069-be58-df81bd51c204 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:45.607078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3666de-57df-4069-be58-df81bd51c204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:45.672655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-af3666de-57df-4069-be58-df81bd51c204 None None] MFA Rules not processed for user `9f28b55f14b8488a9f23c276a094a5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:06:45.673884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-af3666de-57df-4069-be58-df81bd51c204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:45.726540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:45.770839 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 663/1316] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:06:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:06:46.063353 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766085a1-4916-4d4b-b58c-f18f9e8c5b1d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:06:46.064890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-766085a1-4916-4d4b-b58c-f18f9e8c5b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:46.075186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:46.171273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766085a1-4916-4d4b-b58c-f18f9e8c5b1d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:06:46.247933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a3fd4-8d64-41e3-ab94-76273bf87761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:06:46.248545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a3fd4-8d64-41e3-ab94-76273bf87761 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:46.248823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a3fd4-8d64-41e3-ab94-76273bf87761 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:46.249092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a3fd4-8d64-41e3-ab94-76273bf87761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:46.250678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-595a3fd4-8d64-41e3-ab94-76273bf87761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:46.312267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:46.359719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a3fd4-8d64-41e3-ab94-76273bf87761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:06:46.361160 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a3fd4-8d64-41e3-ab94-76273bf87761 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:06:46.382359 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 654/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:06:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:06:49.046561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9d6ad-c1f4-40ce-828e-19d5288ea28e None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:49.047120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9d6ad-c1f4-40ce-828e-19d5288ea28e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:49.047120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9d6ad-c1f4-40ce-828e-19d5288ea28e None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:49.048465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 664/1318] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:06:49.054433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aad07f-b77e-4869-95aa-32e25d635fb0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:49.054844 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aad07f-b77e-4869-95aa-32e25d635fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:49.054982 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aad07f-b77e-4869-95aa-32e25d635fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:49.126145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-54aad07f-b77e-4869-95aa-32e25d635fb0 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:06:49.127385 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-54aad07f-b77e-4869-95aa-32e25d635fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:49.222309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 655/1319] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:06:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:06:49.510745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9d6ad-c1f4-40ce-828e-19d5288ea28e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:06:49.512329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f5c9d6ad-c1f4-40ce-828e-19d5288ea28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:49.616919 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9d6ad-c1f4-40ce-828e-19d5288ea28e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:06:49.690330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc65aba-54f4-4e54-b752-0e9cf8572714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:06:49.690935 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc65aba-54f4-4e54-b752-0e9cf8572714 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:49.691244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc65aba-54f4-4e54-b752-0e9cf8572714 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:49.693165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc65aba-54f4-4e54-b752-0e9cf8572714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:49.693338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1fc65aba-54f4-4e54-b752-0e9cf8572714 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:49.700669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:49.803129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc65aba-54f4-4e54-b752-0e9cf8572714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:06:49.804477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc65aba-54f4-4e54-b752-0e9cf8572714 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:06:49.822634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 665/1320] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:06:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:06:52.680221 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8113343-7cba-4f87-8c19-1427cff1a94b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:52.680754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8113343-7cba-4f87-8c19-1427cff1a94b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:52.680754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8113343-7cba-4f87-8c19-1427cff1a94b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:52.682130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 656/1321] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:06:52.687624 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae9b46-3b8f-4e5c-a7c4-f4d3ca01173e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:52.688026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae9b46-3b8f-4e5c-a7c4-f4d3ca01173e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:52.688280 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae9b46-3b8f-4e5c-a7c4-f4d3ca01173e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:52.764337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-aeae9b46-3b8f-4e5c-a7c4-f4d3ca01173e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:06:52.765586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aeae9b46-3b8f-4e5c-a7c4-f4d3ca01173e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:52.829497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:52.872168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 666/1322] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:06:53.158868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8113343-7cba-4f87-8c19-1427cff1a94b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:06:53.161278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a8113343-7cba-4f87-8c19-1427cff1a94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:53.182602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:53.268058 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8113343-7cba-4f87-8c19-1427cff1a94b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:06:53.343486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779c0f2-c114-4c70-b898-119b2f4f1371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:06:53.344104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779c0f2-c114-4c70-b898-119b2f4f1371 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:53.344383 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779c0f2-c114-4c70-b898-119b2f4f1371 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:53.344655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779c0f2-c114-4c70-b898-119b2f4f1371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:53.346247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c779c0f2-c114-4c70-b898-119b2f4f1371 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:53.419016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:53.458732 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779c0f2-c114-4c70-b898-119b2f4f1371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:06:53.460232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779c0f2-c114-4c70-b898-119b2f4f1371 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:06:53.478046 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 657/1323] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:06:55.703026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba28a0f-94a6-41f9-8a17-37fdf31f40aa None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:55.703605 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba28a0f-94a6-41f9-8a17-37fdf31f40aa None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:55.703705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba28a0f-94a6-41f9-8a17-37fdf31f40aa None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:55.705200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 667/1324] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:06:55.711305 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bb5b1-34dd-4c81-8541-36f3d583d1e1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:55.711740 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bb5b1-34dd-4c81-8541-36f3d583d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:55.711866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bb5b1-34dd-4c81-8541-36f3d583d1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:55.778227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-780bb5b1-34dd-4c81-8541-36f3d583d1e1 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:06:55.779492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-780bb5b1-34dd-4c81-8541-36f3d583d1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:55.886322 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 658/1325] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:06:56.167663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba28a0f-94a6-41f9-8a17-37fdf31f40aa None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:06:56.169458 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0ba28a0f-94a6-41f9-8a17-37fdf31f40aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:56.275439 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba28a0f-94a6-41f9-8a17-37fdf31f40aa None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:06:56.351200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441fd20-5f30-49a0-82d3-3b93b1cdd236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:06:56.351814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441fd20-5f30-49a0-82d3-3b93b1cdd236 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:56.352094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441fd20-5f30-49a0-82d3-3b93b1cdd236 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:56.352364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441fd20-5f30-49a0-82d3-3b93b1cdd236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:56.353974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2441fd20-5f30-49a0-82d3-3b93b1cdd236 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:56.371001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:56.466223 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441fd20-5f30-49a0-82d3-3b93b1cdd236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:06:56.467715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441fd20-5f30-49a0-82d3-3b93b1cdd236 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:06:56.486648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 668/1326] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:06:58.791244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067de0b-2549-490c-82c0-af73e5e60a80 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:58.791748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067de0b-2549-490c-82c0-af73e5e60a80 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:58.792014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067de0b-2549-490c-82c0-af73e5e60a80 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:58.793407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 659/1327] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:06:58.798912 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ea9c1-6700-4fc7-b7e8-def85fe46816 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:58.799360 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ea9c1-6700-4fc7-b7e8-def85fe46816 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:58.799503 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ea9c1-6700-4fc7-b7e8-def85fe46816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:58.871170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b04ea9c1-6700-4fc7-b7e8-def85fe46816 None None] MFA Rules not processed for user `9f28b55f14b8488a9f23c276a094a5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:06:58.872472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b04ea9c1-6700-4fc7-b7e8-def85fe46816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:58.939398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:58.970863 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 669/1328] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:06:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:06:59.277325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1779db-350a-4d1b-9587-3b8a27d6fb18 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:59.277456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1779db-350a-4d1b-9587-3b8a27d6fb18 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:59.277615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1779db-350a-4d1b-9587-3b8a27d6fb18 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:59.279125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 660/1329] 10.222.0.17 () {54 vars in 965 bytes} [Fri Sep 9 13:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:06:59.285908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48de0c6-bc15-457c-9229-07467714fccf None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:59.286191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48de0c6-bc15-457c-9229-07467714fccf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:59.286524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48de0c6-bc15-457c-9229-07467714fccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:59.366160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f48de0c6-bc15-457c-9229-07467714fccf None None] MFA Rules not processed for user `e12deabe077b4095bfa7e2e8fac61d14`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:06:59.367325 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f48de0c6-bc15-457c-9229-07467714fccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:59.444655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 670/1330] 10.222.0.17 () {58 vars in 1062 bytes} [Fri Sep 9 13:06:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:06:59.453550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1779db-350a-4d1b-9587-3b8a27d6fb18 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:06:59.455085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0a1779db-350a-4d1b-9587-3b8a27d6fb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:59.486832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:59.561299 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1779db-350a-4d1b-9587-3b8a27d6fb18 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:06:59.639886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3fe827-e20c-4f88-b901-8ba11f8054a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:06:59.640425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fe827-e20c-4f88-b901-8ba11f8054a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:59.640704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fe827-e20c-4f88-b901-8ba11f8054a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:59.640974 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fe827-e20c-4f88-b901-8ba11f8054a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:59.642557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2e3fe827-e20c-4f88-b901-8ba11f8054a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:59.720699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:59.753933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3fe827-e20c-4f88-b901-8ba11f8054a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:06:59.755430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3fe827-e20c-4f88-b901-8ba11f8054a5 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:06:59.774264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 661/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:06:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:07:02.559645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec355eb6-ba2c-465e-9dfc-cd162be4886d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:02.560204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec355eb6-ba2c-465e-9dfc-cd162be4886d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:02.560204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec355eb6-ba2c-465e-9dfc-cd162be4886d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:02.561576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 671/1332] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:02.567425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2093ad-dde1-431d-992b-d29bbb2050f1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:02.567830 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2093ad-dde1-431d-992b-d29bbb2050f1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:02.567964 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2093ad-dde1-431d-992b-d29bbb2050f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:02.633932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2a2093ad-dde1-431d-992b-d29bbb2050f1 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:02.635179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2a2093ad-dde1-431d-992b-d29bbb2050f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:02.732080 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 662/1333] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:03.028190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec355eb6-ba2c-465e-9dfc-cd162be4886d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:03.029730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ec355eb6-ba2c-465e-9dfc-cd162be4886d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:03.100706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:03.135618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec355eb6-ba2c-465e-9dfc-cd162be4886d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:03.208910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058b376-a3e0-47e1-85d3-e0ff20ddb818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:03.209516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058b376-a3e0-47e1-85d3-e0ff20ddb818 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:03.209793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058b376-a3e0-47e1-85d3-e0ff20ddb818 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:03.210065 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058b376-a3e0-47e1-85d3-e0ff20ddb818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:03.211684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e058b376-a3e0-47e1-85d3-e0ff20ddb818 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:03.319334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058b376-a3e0-47e1-85d3-e0ff20ddb818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:03.320684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058b376-a3e0-47e1-85d3-e0ff20ddb818 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:03.330486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:03.339717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 672/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:05.539404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114df04-bcd3-4b83-a48d-54f69b4ed1ff None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:05.539910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114df04-bcd3-4b83-a48d-54f69b4ed1ff None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:05.539910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114df04-bcd3-4b83-a48d-54f69b4ed1ff None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:05.541259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 663/1335] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:05.546783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909329ff-a528-40eb-85b4-11871b2b94fb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:05.547239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909329ff-a528-40eb-85b4-11871b2b94fb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:05.547364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909329ff-a528-40eb-85b4-11871b2b94fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:05.613693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-909329ff-a528-40eb-85b4-11871b2b94fb None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:05.614937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-909329ff-a528-40eb-85b4-11871b2b94fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:05.712843 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 673/1336] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:05.999910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114df04-bcd3-4b83-a48d-54f69b4ed1ff None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:06.001489 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2114df04-bcd3-4b83-a48d-54f69b4ed1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:06.041791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:06.106535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114df04-bcd3-4b83-a48d-54f69b4ed1ff None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:06.180166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729d135-045c-4d09-ac25-ad787ef68791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:06.180764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729d135-045c-4d09-ac25-ad787ef68791 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:06.181036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729d135-045c-4d09-ac25-ad787ef68791 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:06.181306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729d135-045c-4d09-ac25-ad787ef68791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:06.182856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8729d135-045c-4d09-ac25-ad787ef68791 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:06.264939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:06.292082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729d135-045c-4d09-ac25-ad787ef68791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:06.293935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729d135-045c-4d09-ac25-ad787ef68791 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:06.312455 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 664/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:08.831701 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04bc35-ab2e-4609-8ca3-ca31aa773953 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:08.832297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04bc35-ab2e-4609-8ca3-ca31aa773953 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:08.832297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04bc35-ab2e-4609-8ca3-ca31aa773953 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:08.833617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 674/1338] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:07:08.840623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2149f1a-7a41-4cb4-befd-6a7c66aa7a01 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:08.841086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2149f1a-7a41-4cb4-befd-6a7c66aa7a01 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:08.841192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2149f1a-7a41-4cb4-befd-6a7c66aa7a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:08.914163 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c2149f1a-7a41-4cb4-befd-6a7c66aa7a01 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:08.915416 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c2149f1a-7a41-4cb4-befd-6a7c66aa7a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:09.013279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 665/1339] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:09.294279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04bc35-ab2e-4609-8ca3-ca31aa773953 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:09.295886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3a04bc35-ab2e-4609-8ca3-ca31aa773953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:09.372190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:09.400891 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04bc35-ab2e-4609-8ca3-ca31aa773953 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:09.474883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1eefa-c77b-448f-b4a3-9d57fc93c62f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:09.475508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1eefa-c77b-448f-b4a3-9d57fc93c62f service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:09.475784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1eefa-c77b-448f-b4a3-9d57fc93c62f service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:09.476051 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1eefa-c77b-448f-b4a3-9d57fc93c62f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:09.477604 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-eea1eefa-c77b-448f-b4a3-9d57fc93c62f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:09.588953 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1eefa-c77b-448f-b4a3-9d57fc93c62f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:09.590302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1eefa-c77b-448f-b4a3-9d57fc93c62f service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:09.608702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 675/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:13.819433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2c374-8a35-4544-be42-e54dcb51e4cb None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:13.819954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2c374-8a35-4544-be42-e54dcb51e4cb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:13.819954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2c374-8a35-4544-be42-e54dcb51e4cb None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:13.821252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 666/1341] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:13.826689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215244a9-e79e-4505-9733-98679839c767 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:13.827127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215244a9-e79e-4505-9733-98679839c767 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:13.827276 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215244a9-e79e-4505-9733-98679839c767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:13.833053 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:13.896738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-215244a9-e79e-4505-9733-98679839c767 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:13.897870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-215244a9-e79e-4505-9733-98679839c767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:13.994384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 676/1342] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:14.284078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2c374-8a35-4544-be42-e54dcb51e4cb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:14.285864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4de2c374-8a35-4544-be42-e54dcb51e4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:14.335154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:14.390702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2c374-8a35-4544-be42-e54dcb51e4cb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:14.466109 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28363259-ee37-4c27-846a-4093f9c39559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:14.466722 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28363259-ee37-4c27-846a-4093f9c39559 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:14.466999 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28363259-ee37-4c27-846a-4093f9c39559 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:14.467301 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28363259-ee37-4c27-846a-4093f9c39559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:14.468875 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-28363259-ee37-4c27-846a-4093f9c39559 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:14.563938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:14.581341 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28363259-ee37-4c27-846a-4093f9c39559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:14.582910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28363259-ee37-4c27-846a-4093f9c39559 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:14.603607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 667/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:17.393179 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8b00e-9c1b-48e8-a1a8-bc4df869acc4 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:17.393705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8b00e-9c1b-48e8-a1a8-bc4df869acc4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:17.394103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8b00e-9c1b-48e8-a1a8-bc4df869acc4 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:17.395418 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 677/1344] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:17.400819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3848b8-b3f8-4120-8a35-01faa3b67322 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:17.401259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3848b8-b3f8-4120-8a35-01faa3b67322 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:17.401356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3848b8-b3f8-4120-8a35-01faa3b67322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:17.468545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8e3848b8-b3f8-4120-8a35-01faa3b67322 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:17.469751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8e3848b8-b3f8-4120-8a35-01faa3b67322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:17.571453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 668/1345] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:17.861248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8b00e-9c1b-48e8-a1a8-bc4df869acc4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:17.863195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-72a8b00e-9c1b-48e8-a1a8-bc4df869acc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:17.961451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:17.978923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8b00e-9c1b-48e8-a1a8-bc4df869acc4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:18.068002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b4220-7949-4234-8278-adde01a375e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:18.068519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b4220-7949-4234-8278-adde01a375e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:18.068793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b4220-7949-4234-8278-adde01a375e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:18.069057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b4220-7949-4234-8278-adde01a375e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:18.070631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-983b4220-7949-4234-8278-adde01a375e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:18.188380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b4220-7949-4234-8278-adde01a375e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:18.190277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b4220-7949-4234-8278-adde01a375e2 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:18.208956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 678/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:22.425824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa43d87-c233-4278-ac21-c2468c5bd694 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:22.426384 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa43d87-c233-4278-ac21-c2468c5bd694 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:22.426384 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa43d87-c233-4278-ac21-c2468c5bd694 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:22.427908 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 669/1347] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:07:22.433199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8038350-41f7-4f58-acdd-213edc918d48 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:22.433675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8038350-41f7-4f58-acdd-213edc918d48 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:22.433772 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8038350-41f7-4f58-acdd-213edc918d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:22.449727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:22.509592 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e8038350-41f7-4f58-acdd-213edc918d48 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:22.510986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e8038350-41f7-4f58-acdd-213edc918d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:22.634999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 679/1348] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:22.917223 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa43d87-c233-4278-ac21-c2468c5bd694 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:22.919024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0fa43d87-c233-4278-ac21-c2468c5bd694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:22.976572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:23.022774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa43d87-c233-4278-ac21-c2468c5bd694 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:23.096463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b4923-07bf-4463-a34a-0ff6b9d334cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:23.097067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b4923-07bf-4463-a34a-0ff6b9d334cc service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:23.097344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b4923-07bf-4463-a34a-0ff6b9d334cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:23.097611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b4923-07bf-4463-a34a-0ff6b9d334cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:23.099267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3a4b4923-07bf-4463-a34a-0ff6b9d334cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:23.204961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:23.209518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b4923-07bf-4463-a34a-0ff6b9d334cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:23.210967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b4923-07bf-4463-a34a-0ff6b9d334cc service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:23.229303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 670/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:28.163181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b209fb-08bf-4c4a-8528-dab49bc9d4fa None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:28.163953 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b209fb-08bf-4c4a-8528-dab49bc9d4fa None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:28.164382 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b209fb-08bf-4c4a-8528-dab49bc9d4fa None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:28.167162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 680/1350] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:07:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:28.178040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc1976-88f2-4d8d-8712-6fda23581827 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:28.178416 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc1976-88f2-4d8d-8712-6fda23581827 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:28.178595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc1976-88f2-4d8d-8712-6fda23581827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:28.332215 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e3cc1976-88f2-4d8d-8712-6fda23581827 None None] MFA Rules not processed for user `dbc7cef7b714463697dc1ee507e60be1`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:28.333500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e3cc1976-88f2-4d8d-8712-6fda23581827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:28.458518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 671/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:07:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:32.633279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=59 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:32.711566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cf097-b7cd-4fd4-b8f2-3a507db8ef21 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:32.711861 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cf097-b7cd-4fd4-b8f2-3a507db8ef21 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:32.712352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cf097-b7cd-4fd4-b8f2-3a507db8ef21 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:32.713715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 681/1352] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:32.718825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6accd6a-2367-4a81-a7e2-f0be2e3976f1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:32.719306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6accd6a-2367-4a81-a7e2-f0be2e3976f1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:32.720214 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6accd6a-2367-4a81-a7e2-f0be2e3976f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:32.776646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:32.802466 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f6accd6a-2367-4a81-a7e2-f0be2e3976f1 None None] MFA Rules not processed for user `dbc7cef7b714463697dc1ee507e60be1`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:32.804573 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f6accd6a-2367-4a81-a7e2-f0be2e3976f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:32.927118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 672/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:07:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:34.166848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a8c68-3aa2-4583-b826-9481e6543cb8 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:34.167387 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a8c68-3aa2-4583-b826-9481e6543cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:34.167387 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a8c68-3aa2-4583-b826-9481e6543cb8 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:34.168955 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 682/1354] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:34.174737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4a4b0-bb2d-4817-830d-ffc598881d8f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:34.175543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4a4b0-bb2d-4817-830d-ffc598881d8f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:34.175822 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4a4b0-bb2d-4817-830d-ffc598881d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:34.255782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-72d4a4b0-bb2d-4817-830d-ffc598881d8f None None] MFA Rules not processed for user `dbc7cef7b714463697dc1ee507e60be1`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:34.257002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-72d4a4b0-bb2d-4817-830d-ffc598881d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:34.305849 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:34.372368 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 673/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:07:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:37.882519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5e497-8411-4a61-b758-9f9f9096a9f1 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:37.883030 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5e497-8411-4a61-b758-9f9f9096a9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:37.883271 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5e497-8411-4a61-b758-9f9f9096a9f1 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:37.884734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 683/1356] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:37.890208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb53d16-40cb-4590-8268-73fe353d71de None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:37.890615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb53d16-40cb-4590-8268-73fe353d71de None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:37.891002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb53d16-40cb-4590-8268-73fe353d71de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:37.962556 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8bb53d16-40cb-4590-8268-73fe353d71de None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:37.963953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8bb53d16-40cb-4590-8268-73fe353d71de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:38.070761 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 674/1357] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:38.370135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b242d-fd11-4ab0-97b7-c0673392d26b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:38.370422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b242d-fd11-4ab0-97b7-c0673392d26b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:38.370698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b242d-fd11-4ab0-97b7-c0673392d26b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:38.372336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 684/1358] 10.222.0.17 () {54 vars in 962 bytes} [Fri Sep 9 13:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:38.380676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37229d37-0ea4-4825-b255-bd897720690e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:38.381006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37229d37-0ea4-4825-b255-bd897720690e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:38.381236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37229d37-0ea4-4825-b255-bd897720690e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:38.401089 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:38.486148 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-37229d37-0ea4-4825-b255-bd897720690e None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:38.487452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-37229d37-0ea4-4825-b255-bd897720690e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:38.627874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 675/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Fri Sep 9 13:07:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:38.635655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b242d-fd11-4ab0-97b7-c0673392d26b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:38.637326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-248b242d-fd11-4ab0-97b7-c0673392d26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:38.795520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b242d-fd11-4ab0-97b7-c0673392d26b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:38.882330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:38.946993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b68a69-cf28-407d-8810-6a9d2c4d56fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:38.947798 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b68a69-cf28-407d-8810-6a9d2c4d56fa service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:38.947899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b68a69-cf28-407d-8810-6a9d2c4d56fa service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:38.948326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b68a69-cf28-407d-8810-6a9d2c4d56fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:38.950067 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-83b68a69-cf28-407d-8810-6a9d2c4d56fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:39.051347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b68a69-cf28-407d-8810-6a9d2c4d56fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:39.052956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b68a69-cf28-407d-8810-6a9d2c4d56fa service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:39.072031 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 685/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:41.468230 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a61b9-e302-43d0-aee1-45d881c9f817 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:41.469350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a61b9-e302-43d0-aee1-45d881c9f817 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:41.469916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a61b9-e302-43d0-aee1-45d881c9f817 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:41.473384 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 676/1361] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:41 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:41.478774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692b020-90b7-439c-9450-b7eb0c0bde42 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:41.479260 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692b020-90b7-439c-9450-b7eb0c0bde42 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:41.479487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692b020-90b7-439c-9450-b7eb0c0bde42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:41.552417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f692b020-90b7-439c-9450-b7eb0c0bde42 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:41.553660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f692b020-90b7-439c-9450-b7eb0c0bde42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:41.598666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:41.664414 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 686/1362] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:41.955302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a61b9-e302-43d0-aee1-45d881c9f817 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:41.956869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e93a61b9-e302-43d0-aee1-45d881c9f817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:42.035548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:42.139387 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a61b9-e302-43d0-aee1-45d881c9f817 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:42.265425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12347c-8c28-47fc-8488-14152c5d5ab3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:42.266207 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12347c-8c28-47fc-8488-14152c5d5ab3 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:42.266504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12347c-8c28-47fc-8488-14152c5d5ab3 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:42.266796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12347c-8c28-47fc-8488-14152c5d5ab3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:42.268440 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cc12347c-8c28-47fc-8488-14152c5d5ab3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:42.291470 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:42.362911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12347c-8c28-47fc-8488-14152c5d5ab3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:42.364576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12347c-8c28-47fc-8488-14152c5d5ab3 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:42.383958 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 677/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:44.889100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952aa8e-1579-4dd5-bf9d-f44c7b6173b4 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:44.889621 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952aa8e-1579-4dd5-bf9d-f44c7b6173b4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:44.889621 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952aa8e-1579-4dd5-bf9d-f44c7b6173b4 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:44.891650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 687/1364] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:44.897044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a11b3-e18e-433c-bbfc-336aa6786a32 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:44.897479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a11b3-e18e-433c-bbfc-336aa6786a32 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:44.897625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a11b3-e18e-433c-bbfc-336aa6786a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:44.968036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4c3a11b3-e18e-433c-bbfc-336aa6786a32 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:44.969283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4c3a11b3-e18e-433c-bbfc-336aa6786a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:45.074881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:45.077594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 678/1365] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:45.369810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cfdf5-c10a-4f77-94dc-84bd717c2dbe None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:45.370113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cfdf5-c10a-4f77-94dc-84bd717c2dbe None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:45.370410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cfdf5-c10a-4f77-94dc-84bd717c2dbe None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:45.371896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 688/1366] 10.222.0.17 () {54 vars in 962 bytes} [Fri Sep 9 13:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:45.378498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578e415-1b08-400d-b89e-2f6bf88d2a6f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:45.378859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578e415-1b08-400d-b89e-2f6bf88d2a6f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:45.379247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578e415-1b08-400d-b89e-2f6bf88d2a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:45.460841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3578e415-1b08-400d-b89e-2f6bf88d2a6f None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:45.462060 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3578e415-1b08-400d-b89e-2f6bf88d2a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:45.585681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 679/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Fri Sep 9 13:07:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:45.595431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cfdf5-c10a-4f77-94dc-84bd717c2dbe None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:45.595826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-952cfdf5-c10a-4f77-94dc-84bd717c2dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:45.757953 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cfdf5-c10a-4f77-94dc-84bd717c2dbe None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:45.784647 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:45.887017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335d787-3f75-4474-89a0-e4f3bdc42eea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:45.887592 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335d787-3f75-4474-89a0-e4f3bdc42eea service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:45.887870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335d787-3f75-4474-89a0-e4f3bdc42eea service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:45.888137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335d787-3f75-4474-89a0-e4f3bdc42eea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:45.889730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e335d787-3f75-4474-89a0-e4f3bdc42eea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:45.991232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335d787-3f75-4474-89a0-e4f3bdc42eea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:45.992900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335d787-3f75-4474-89a0-e4f3bdc42eea service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:46.014985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 689/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:46.108895 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762b2a0-f617-4d9f-879d-1f3a1fcdf845 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:46.110437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762b2a0-f617-4d9f-879d-1f3a1fcdf845 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:46.111010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762b2a0-f617-4d9f-879d-1f3a1fcdf845 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:46.113070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 680/1369] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:46.127358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35284b6-4ecf-4324-b065-3f04622d7737 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:46.127358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35284b6-4ecf-4324-b065-3f04622d7737 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:46.127358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35284b6-4ecf-4324-b065-3f04622d7737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:46.158920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:46.206468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b35284b6-4ecf-4324-b065-3f04622d7737 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:46.207853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b35284b6-4ecf-4324-b065-3f04622d7737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:46.322801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 690/1370] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:46.660079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448cd2e-b969-4982-8187-16023a566790 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:46.660079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448cd2e-b969-4982-8187-16023a566790 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:46.660383 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448cd2e-b969-4982-8187-16023a566790 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:46.662032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 681/1371] 10.222.0.17 () {54 vars in 964 bytes} [Fri Sep 9 13:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:46.666716 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcaf23-57ee-4e7e-a79b-c6ee2a8f5aab None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:46.667284 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcaf23-57ee-4e7e-a79b-c6ee2a8f5aab None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:46.667560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcaf23-57ee-4e7e-a79b-c6ee2a8f5aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:46.755734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-57fcaf23-57ee-4e7e-a79b-c6ee2a8f5aab None None] MFA Rules not processed for user `ea77fa8f5b754a2287f0639717f6e57a`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:46.757065 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-57fcaf23-57ee-4e7e-a79b-c6ee2a8f5aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:46.808379 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:46.840777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 691/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Fri Sep 9 13:07:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:46.849207 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448cd2e-b969-4982-8187-16023a566790 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:46.850702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2448cd2e-b969-4982-8187-16023a566790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:46.905105 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:46.967314 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448cd2e-b969-4982-8187-16023a566790 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:47.064814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb89bf9-9091-43de-905a-daa54b53b217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:47.065431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb89bf9-9091-43de-905a-daa54b53b217 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:47.065686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb89bf9-9091-43de-905a-daa54b53b217 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:47.066024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb89bf9-9091-43de-905a-daa54b53b217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:47.067726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-acb89bf9-9091-43de-905a-daa54b53b217 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:47.175811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:47.189298 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb89bf9-9091-43de-905a-daa54b53b217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:47.190785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb89bf9-9091-43de-905a-daa54b53b217 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:47.211488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 682/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:48.269945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200fb4a-4d7c-4e80-a575-cb8ba92e2ddd None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:48.270477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200fb4a-4d7c-4e80-a575-cb8ba92e2ddd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:48.270477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200fb4a-4d7c-4e80-a575-cb8ba92e2ddd None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:48.271884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 692/1374] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:48.277388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04469a53-21f5-4962-b2ba-1443a342c6f5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:48.277858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04469a53-21f5-4962-b2ba-1443a342c6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:48.277967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04469a53-21f5-4962-b2ba-1443a342c6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:48.344717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-04469a53-21f5-4962-b2ba-1443a342c6f5 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:48.345937 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-04469a53-21f5-4962-b2ba-1443a342c6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:48.445154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 683/1375] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:48.739611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200fb4a-4d7c-4e80-a575-cb8ba92e2ddd None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:48.741151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d200fb4a-4d7c-4e80-a575-cb8ba92e2ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:48.901267 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200fb4a-4d7c-4e80-a575-cb8ba92e2ddd None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:48.956353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:49.024029 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23aac6e-dccc-45f2-ade3-4a600e96f1ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:49.024671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23aac6e-dccc-45f2-ade3-4a600e96f1ea service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:49.024954 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23aac6e-dccc-45f2-ade3-4a600e96f1ea service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:49.025228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23aac6e-dccc-45f2-ade3-4a600e96f1ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:49.026828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c23aac6e-dccc-45f2-ade3-4a600e96f1ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:49.121646 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23aac6e-dccc-45f2-ade3-4a600e96f1ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:49.123331 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23aac6e-dccc-45f2-ade3-4a600e96f1ea service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:49.146393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 693/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:51.376051 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c9d69-9fce-4bb4-b034-80b6d37bb4fb None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:51.376586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c9d69-9fce-4bb4-b034-80b6d37bb4fb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:51.376586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c9d69-9fce-4bb4-b034-80b6d37bb4fb None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:51.378085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 684/1377] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:51.383362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f603eb-1110-4da3-82ab-17e152bb4be5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:51.383778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f603eb-1110-4da3-82ab-17e152bb4be5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:51.383946 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f603eb-1110-4da3-82ab-17e152bb4be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:51.452080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-26f603eb-1110-4da3-82ab-17e152bb4be5 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:51.453496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-26f603eb-1110-4da3-82ab-17e152bb4be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:51.460187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:51.555032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 694/1378] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:51.845142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c9d69-9fce-4bb4-b034-80b6d37bb4fb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:51.846664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7b1c9d69-9fce-4bb4-b034-80b6d37bb4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:51.912559 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:52.007609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c9d69-9fce-4bb4-b034-80b6d37bb4fb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:52.130203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2fc03-c6f6-4f67-aeb8-cbe24f273aa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:52.130712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2fc03-c6f6-4f67-aeb8-cbe24f273aa0 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:52.130995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2fc03-c6f6-4f67-aeb8-cbe24f273aa0 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:52.131296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2fc03-c6f6-4f67-aeb8-cbe24f273aa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:52.132906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a1a2fc03-c6f6-4f67-aeb8-cbe24f273aa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:52.153235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:52.229867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2fc03-c6f6-4f67-aeb8-cbe24f273aa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:52.231511 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2fc03-c6f6-4f67-aeb8-cbe24f273aa0 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:52.251094 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 685/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:54.477978 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca68805-c042-490d-a2ba-2786c3beea98 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:54.478499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca68805-c042-490d-a2ba-2786c3beea98 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:54.478684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca68805-c042-490d-a2ba-2786c3beea98 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:54.480391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 695/1380] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:54.486341 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b6058-d215-4e79-9602-2a0c3d9bd1e4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:54.486798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b6058-d215-4e79-9602-2a0c3d9bd1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:54.486905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b6058-d215-4e79-9602-2a0c3d9bd1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:54.554608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e70b6058-d215-4e79-9602-2a0c3d9bd1e4 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:54.555860 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e70b6058-d215-4e79-9602-2a0c3d9bd1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:54.677459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:54.678134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 686/1381] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:54.977165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca68805-c042-490d-a2ba-2786c3beea98 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:54.978753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dca68805-c042-490d-a2ba-2786c3beea98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:55.125375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:55.138556 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca68805-c042-490d-a2ba-2786c3beea98 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:55.257004 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f2a01-fb28-47d1-895d-5d236997db0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:55.257536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f2a01-fb28-47d1-895d-5d236997db0d service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:55.257812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f2a01-fb28-47d1-895d-5d236997db0d service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:55.258079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f2a01-fb28-47d1-895d-5d236997db0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:55.259693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-713f2a01-fb28-47d1-895d-5d236997db0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:55.359659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f2a01-fb28-47d1-895d-5d236997db0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:55.361451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f2a01-fb28-47d1-895d-5d236997db0d service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:55.372288 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:55.382053 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 696/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:57.599087 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b833d-d7bb-4d8d-9c96-0ecfaab355aa None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:57.599654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b833d-d7bb-4d8d-9c96-0ecfaab355aa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:57.599805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b833d-d7bb-4d8d-9c96-0ecfaab355aa None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:57.601558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 687/1383] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:07:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:07:57.607101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf97db-3a26-4892-b914-1865e0b17ff9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:57.607904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf97db-3a26-4892-b914-1865e0b17ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:57.609308 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf97db-3a26-4892-b914-1865e0b17ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:57.682234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-eebf97db-3a26-4892-b914-1865e0b17ff9 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:07:57.683491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-eebf97db-3a26-4892-b914-1865e0b17ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:57.783391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 697/1384] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:58.078665 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b833d-d7bb-4d8d-9c96-0ecfaab355aa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:58.080711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b17b833d-d7bb-4d8d-9c96-0ecfaab355aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:58.258838 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b833d-d7bb-4d8d-9c96-0ecfaab355aa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:58.337753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:58.389912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c02a2-3e18-4999-933f-2e83fba54a07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:58.390557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c02a2-3e18-4999-933f-2e83fba54a07 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:58.391019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c02a2-3e18-4999-933f-2e83fba54a07 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:58.391342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c02a2-3e18-4999-933f-2e83fba54a07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:58.393683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a57c02a2-3e18-4999-933f-2e83fba54a07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:58.503613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c02a2-3e18-4999-933f-2e83fba54a07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:58.505157 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c02a2-3e18-4999-933f-2e83fba54a07 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:58.526007 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 688/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:00.720512 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77a72b-9054-4303-8a30-3b96bc590713 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:00.721216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77a72b-9054-4303-8a30-3b96bc590713 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:00.721216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77a72b-9054-4303-8a30-3b96bc590713 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:00.722285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 698/1386] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:08:00.727689 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0ebbb-3cd9-4171-9227-00de2fd47f6a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:00.728228 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0ebbb-3cd9-4171-9227-00de2fd47f6a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:00.729573 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0ebbb-3cd9-4171-9227-00de2fd47f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:00.798368 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b0e0ebbb-3cd9-4171-9227-00de2fd47f6a None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:00.799641 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b0e0ebbb-3cd9-4171-9227-00de2fd47f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:00.834991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:00.901445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 689/1387] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:01.190069 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77a72b-9054-4303-8a30-3b96bc590713 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:01.191654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5f77a72b-9054-4303-8a30-3b96bc590713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:01.269049 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:01.340327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77a72b-9054-4303-8a30-3b96bc590713 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:01.454456 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bac67c-03d9-49af-b95e-3deec4388df0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:01.454991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bac67c-03d9-49af-b95e-3deec4388df0 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:01.455298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bac67c-03d9-49af-b95e-3deec4388df0 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:01.455574 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bac67c-03d9-49af-b95e-3deec4388df0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:01.457154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-26bac67c-03d9-49af-b95e-3deec4388df0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:01.497826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:01.555236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bac67c-03d9-49af-b95e-3deec4388df0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:01.556883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bac67c-03d9-49af-b95e-3deec4388df0 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:01.579793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 699/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:02.958485 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb2c5f-77d8-4b67-84a4-b7804df92bc2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:02.959094 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb2c5f-77d8-4b67-84a4-b7804df92bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:02.959094 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb2c5f-77d8-4b67-84a4-b7804df92bc2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:02.960458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 690/1389] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:02.965843 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d208d-a9b2-4c03-824a-51d664d79d41 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:02.966274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d208d-a9b2-4c03-824a-51d664d79d41 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:02.966400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d208d-a9b2-4c03-824a-51d664d79d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:03.033671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a71d208d-a9b2-4c03-824a-51d664d79d41 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:03.034898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a71d208d-a9b2-4c03-824a-51d664d79d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:03.132316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 700/1390] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:03.436819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7d190-8689-4f45-b68d-6fc14b7ebad9 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:03.437176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7d190-8689-4f45-b68d-6fc14b7ebad9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:03.437521 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7d190-8689-4f45-b68d-6fc14b7ebad9 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:03.439533 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 691/1391] 10.222.0.17 () {54 vars in 966 bytes} [Fri Sep 9 13:08:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:03.444733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf9a99-9442-4c65-8808-d95d67ee8e1e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:03.445022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf9a99-9442-4c65-8808-d95d67ee8e1e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:03.445295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf9a99-9442-4c65-8808-d95d67ee8e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:03.461534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:03.523000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2dcf9a99-9442-4c65-8808-d95d67ee8e1e None None] MFA Rules not processed for user `a7ef6cd64182474787134e3227beb3aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:03.524250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2dcf9a99-9442-4c65-8808-d95d67ee8e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:03.608155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 701/1392] 10.222.0.17 () {58 vars in 1063 bytes} [Fri Sep 9 13:08:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:03.615727 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7d190-8689-4f45-b68d-6fc14b7ebad9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:03.617241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-82b7d190-8689-4f45-b68d-6fc14b7ebad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:03.728301 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7d190-8689-4f45-b68d-6fc14b7ebad9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:03.785189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f32e5-05c8-40c6-9f4f-a57917d8be20 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:03.785471 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f32e5-05c8-40c6-9f4f-a57917d8be20 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:03.785747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f32e5-05c8-40c6-9f4f-a57917d8be20 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:03.787140 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 702/1393] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:08:03.792015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:03.792149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b9d3e-32b7-4dac-afaa-152e554a823d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:03.792408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b9d3e-32b7-4dac-afaa-152e554a823d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:03.792688 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b9d3e-32b7-4dac-afaa-152e554a823d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:03.809034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98360acc-a8ce-447b-b0bb-303cb4beac2e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ef6cd64182474787134e3227beb3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:03.809660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98360acc-a8ce-447b-b0bb-303cb4beac2e service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:03.809936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98360acc-a8ce-447b-b0bb-303cb4beac2e service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:03.810212 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98360acc-a8ce-447b-b0bb-303cb4beac2e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:03.811861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-98360acc-a8ce-447b-b0bb-303cb4beac2e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:03.861760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c66b9d3e-32b7-4dac-afaa-152e554a823d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:03.863010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c66b9d3e-32b7-4dac-afaa-152e554a823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:03.919309 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:03.926110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98360acc-a8ce-447b-b0bb-303cb4beac2e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:03.927833 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98360acc-a8ce-447b-b0bb-303cb4beac2e service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:03.947174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 692/1394] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:03.963587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 703/1395] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:08:04.257842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98360acc-a8ce-447b-b0bb-303cb4beac2e service glance] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:04.259065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-98360acc-a8ce-447b-b0bb-303cb4beac2e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:04.376456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:04.384528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98360acc-a8ce-447b-b0bb-303cb4beac2e service glance] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:04.530323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0940a9-dcb4-479b-a67d-94d48c9aea8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:04.530890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0940a9-dcb4-479b-a67d-94d48c9aea8d service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:04.531267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0940a9-dcb4-479b-a67d-94d48c9aea8d service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:04.531547 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0940a9-dcb4-479b-a67d-94d48c9aea8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:04.534257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ee0940a9-dcb4-479b-a67d-94d48c9aea8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:04.661966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0940a9-dcb4-479b-a67d-94d48c9aea8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:04.663622 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0940a9-dcb4-479b-a67d-94d48c9aea8d service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:04.683357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:04.691148 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 693/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:04.716171 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd4d53-fd3d-458d-a518-f04975286aec None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:04.716634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd4d53-fd3d-458d-a518-f04975286aec None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:04.716839 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd4d53-fd3d-458d-a518-f04975286aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:04.794067 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dbfd4d53-fd3d-458d-a518-f04975286aec None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:04.795276 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dbfd4d53-fd3d-458d-a518-f04975286aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:04.894941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 704/1397] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:08:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:04.943898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fef4a-c47d-4caa-99ab-6d981d167e4a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:04.944285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fef4a-c47d-4caa-99ab-6d981d167e4a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:04.944629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fef4a-c47d-4caa-99ab-6d981d167e4a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:04.946124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 694/1398] 10.222.0.17 () {54 vars in 943 bytes} [Fri Sep 9 13:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:04.952733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c5382-2af5-4809-be1b-dc40f6561a70 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:04.953022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c5382-2af5-4809-be1b-dc40f6561a70 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:04.953326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c5382-2af5-4809-be1b-dc40f6561a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:05.014746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:05.034519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ed6c5382-2af5-4809-be1b-dc40f6561a70 None None] MFA Rules not processed for user `c361a3e83f8e45c2b4d28f97f0a9d069`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:05.035759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ed6c5382-2af5-4809-be1b-dc40f6561a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:05.143448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 705/1399] 10.222.0.17 () {58 vars in 1040 bytes} [Fri Sep 9 13:08:04 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:05.152809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fef4a-c47d-4caa-99ab-6d981d167e4a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:05.154337 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-423fef4a-c47d-4caa-99ab-6d981d167e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:05.300018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fef4a-c47d-4caa-99ab-6d981d167e4a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:05.386323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:05.406002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d39a1e-c5c3-40b3-8760-c39325f941dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:05.406539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d39a1e-c5c3-40b3-8760-c39325f941dc service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:05.406818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d39a1e-c5c3-40b3-8760-c39325f941dc service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:05.407134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d39a1e-c5c3-40b3-8760-c39325f941dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:05.409698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-05d39a1e-c5c3-40b3-8760-c39325f941dc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:05.494320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d39a1e-c5c3-40b3-8760-c39325f941dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:05.495954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d39a1e-c5c3-40b3-8760-c39325f941dc service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:05.504158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 695/1400] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:08:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:08:06.937174 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994190ca-0941-438b-8b17-b39d2dc9e4c4 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:06.937704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994190ca-0941-438b-8b17-b39d2dc9e4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:06.937704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994190ca-0941-438b-8b17-b39d2dc9e4c4 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:06.943164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 706/1401] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:08:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:06.944599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e044f9-9fd9-41fb-8291-d17f1b14cb8c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:06.945045 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e044f9-9fd9-41fb-8291-d17f1b14cb8c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:06.945140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e044f9-9fd9-41fb-8291-d17f1b14cb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:07.020935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-51e044f9-9fd9-41fb-8291-d17f1b14cb8c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:07.022252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-51e044f9-9fd9-41fb-8291-d17f1b14cb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:07.101687 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:07.132447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 696/1402] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:08:07.427175 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994190ca-0941-438b-8b17-b39d2dc9e4c4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:07.428944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-994190ca-0941-438b-8b17-b39d2dc9e4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:07.547930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:07.601229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994190ca-0941-438b-8b17-b39d2dc9e4c4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:07.749855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a78224-b6f4-40e4-b723-24c17a3d0803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:07.750761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a78224-b6f4-40e4-b723-24c17a3d0803 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:07.751037 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a78224-b6f4-40e4-b723-24c17a3d0803 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:07.751337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a78224-b6f4-40e4-b723-24c17a3d0803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:07.752991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e0a78224-b6f4-40e4-b723-24c17a3d0803 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:07.837544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:07.858524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a78224-b6f4-40e4-b723-24c17a3d0803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:07.860188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a78224-b6f4-40e4-b723-24c17a3d0803 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:07.881378 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 707/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:10.147493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b467edc-6343-4d4b-a236-8d57df0be77b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:10.148096 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b467edc-6343-4d4b-a236-8d57df0be77b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:10.148351 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b467edc-6343-4d4b-a236-8d57df0be77b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:10.149716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 697/1404] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:10.155263 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a1517-e6b7-46a5-8c87-4022f792d403 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:10.156077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a1517-e6b7-46a5-8c87-4022f792d403 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:10.156361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a1517-e6b7-46a5-8c87-4022f792d403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:10.234053 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7f7a1517-e6b7-46a5-8c87-4022f792d403 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:10.235667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7f7a1517-e6b7-46a5-8c87-4022f792d403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:10.348629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 708/1405] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:10.640997 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b467edc-6343-4d4b-a236-8d57df0be77b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:10.642520 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6b467edc-6343-4d4b-a236-8d57df0be77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:10.815639 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b467edc-6343-4d4b-a236-8d57df0be77b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:10.877871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:10.945100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a12db-3cd2-4e64-8bb3-08f389222b9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:10.945720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a12db-3cd2-4e64-8bb3-08f389222b9a service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:10.946000 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a12db-3cd2-4e64-8bb3-08f389222b9a service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:10.946270 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a12db-3cd2-4e64-8bb3-08f389222b9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:10.947921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-467a12db-3cd2-4e64-8bb3-08f389222b9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:11.054787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a12db-3cd2-4e64-8bb3-08f389222b9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:11.056566 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a12db-3cd2-4e64-8bb3-08f389222b9a service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:11.076587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 698/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:13.304851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af84fb-3e39-41ca-a3fa-822f57c40ba7 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:13.305430 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af84fb-3e39-41ca-a3fa-822f57c40ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:13.305430 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af84fb-3e39-41ca-a3fa-822f57c40ba7 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:13.310790 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 709/1407] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:08:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:13.312370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e8eb0-df5f-4a4b-bded-d60edc65884f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:13.312820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e8eb0-df5f-4a4b-bded-d60edc65884f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:13.312913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e8eb0-df5f-4a4b-bded-d60edc65884f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:13.383446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-af5e8eb0-df5f-4a4b-bded-d60edc65884f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:13.385053 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-af5e8eb0-df5f-4a4b-bded-d60edc65884f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:13.400627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:13.499124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 699/1408] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:08:13.795114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af84fb-3e39-41ca-a3fa-822f57c40ba7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:13.796656 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-40af84fb-3e39-41ca-a3fa-822f57c40ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:13.854720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:13.966138 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af84fb-3e39-41ca-a3fa-822f57c40ba7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:14.234150 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937139db-a047-4471-ac75-9bdfc6f90767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:14.234853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937139db-a047-4471-ac75-9bdfc6f90767 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:14.235261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937139db-a047-4471-ac75-9bdfc6f90767 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:14.235552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937139db-a047-4471-ac75-9bdfc6f90767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:14.237606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-937139db-a047-4471-ac75-9bdfc6f90767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:14.248273 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:14.367973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937139db-a047-4471-ac75-9bdfc6f90767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:14.369604 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937139db-a047-4471-ac75-9bdfc6f90767 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:14.390644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 710/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:23.493733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=37 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:24.387486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=41 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:28.555887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b8f77-db2c-4829-9a4f-2d64c9c0f5b7 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:28.556478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b8f77-db2c-4829-9a4f-2d64c9c0f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:28.556478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b8f77-db2c-4829-9a4f-2d64c9c0f5b7 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:28.558203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 700/1410] 10.222.0.17 () {54 vars in 918 bytes} [Fri Sep 9 13:08:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:28.563522 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53bd4c-a741-4b55-bfa4-886d96d0d699 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:28.563928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53bd4c-a741-4b55-bfa4-886d96d0d699 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:28.564063 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53bd4c-a741-4b55-bfa4-886d96d0d699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:28.642345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ca53bd4c-a741-4b55-bfa4-886d96d0d699 None None] MFA Rules not processed for user `dbc7cef7b714463697dc1ee507e60be1`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:28.643614 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ca53bd4c-a741-4b55-bfa4-886d96d0d699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:28.748648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 711/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Fri Sep 9 13:08:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:08:28.763463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cca5f8-6240-4fb4-bd8f-ea2ed07ad4cb None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:28.763752 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cca5f8-6240-4fb4-bd8f-ea2ed07ad4cb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:28.764028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cca5f8-6240-4fb4-bd8f-ea2ed07ad4cb None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:28.765370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 701/1412] 10.222.0.17 () {54 vars in 918 bytes} [Fri Sep 9 13:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:28.769945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd13780-06ba-43bd-a4a8-7133f52187cb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:28.770232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd13780-06ba-43bd-a4a8-7133f52187cb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:28.770508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd13780-06ba-43bd-a4a8-7133f52187cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:28.843186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:28.846426 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5bd13780-06ba-43bd-a4a8-7133f52187cb None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:28.848440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5bd13780-06ba-43bd-a4a8-7133f52187cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:28.967676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 712/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Fri Sep 9 13:08:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:08:31.553880 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d6813-941f-4cce-a492-047f889dcb43 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:31.554467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d6813-941f-4cce-a492-047f889dcb43 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:31.554467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d6813-941f-4cce-a492-047f889dcb43 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:31.556097 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:31.561586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cccca-33cf-41f4-bf66-008d576bd3de None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:31.562013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cccca-33cf-41f4-bf66-008d576bd3de None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:31.562151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cccca-33cf-41f4-bf66-008d576bd3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:31.629506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3d0cccca-33cf-41f4-bf66-008d576bd3de None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:31.630715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3d0cccca-33cf-41f4-bf66-008d576bd3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:31.685799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:31.726984 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:32.018372 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d6813-941f-4cce-a492-047f889dcb43 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:32.019980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-660d6813-941f-4cce-a492-047f889dcb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:32.165935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d6813-941f-4cce-a492-047f889dcb43 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:32.239090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:32.278891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58607b6-e798-419a-84e8-7891148963c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:32.279465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58607b6-e798-419a-84e8-7891148963c4 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:32.279743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58607b6-e798-419a-84e8-7891148963c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:32.280012 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58607b6-e798-419a-84e8-7891148963c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:32.281612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e58607b6-e798-419a-84e8-7891148963c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:32.371659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58607b6-e798-419a-84e8-7891148963c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:32.373241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58607b6-e798-419a-84e8-7891148963c4 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:32.390805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 703/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:33.527021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3919a12-a18c-4084-8c2d-1c0950b43111 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:33.527615 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3919a12-a18c-4084-8c2d-1c0950b43111 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:33.527615 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3919a12-a18c-4084-8c2d-1c0950b43111 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:33.529274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 13:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:33.535706 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec5d2b-361b-4c27-8cf7-2a217fc5aaa8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:33.536156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec5d2b-361b-4c27-8cf7-2a217fc5aaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:33.536254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec5d2b-361b-4c27-8cf7-2a217fc5aaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:33.613598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f4ec5d2b-361b-4c27-8cf7-2a217fc5aaa8 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:33.614827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f4ec5d2b-361b-4c27-8cf7-2a217fc5aaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:33.642362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:33.712474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 13:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:33.864198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3919a12-a18c-4084-8c2d-1c0950b43111 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:33.865630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b3919a12-a18c-4084-8c2d-1c0950b43111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:34.014534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3919a12-a18c-4084-8c2d-1c0950b43111 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:34.055361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:34.130209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad05802-589a-4d6b-95fa-7ca59fea24a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:34.130730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad05802-589a-4d6b-95fa-7ca59fea24a3 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:34.131006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad05802-589a-4d6b-95fa-7ca59fea24a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:34.131371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad05802-589a-4d6b-95fa-7ca59fea24a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:34.132947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dad05802-589a-4d6b-95fa-7ca59fea24a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:34.222379 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad05802-589a-4d6b-95fa-7ca59fea24a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:34.224050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad05802-589a-4d6b-95fa-7ca59fea24a3 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:34.242507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 715/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:35.379683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea11b9-995b-40a4-8561-df4977593c0a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:35.380231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea11b9-995b-40a4-8561-df4977593c0a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:35.380231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea11b9-995b-40a4-8561-df4977593c0a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:35.381584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 13:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:35.387750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf8280-69fb-4c2d-9148-f5e26fcce57b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:35.388830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf8280-69fb-4c2d-9148-f5e26fcce57b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:35.389100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf8280-69fb-4c2d-9148-f5e26fcce57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:35.442875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:35.464380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9ddf8280-69fb-4c2d-9148-f5e26fcce57b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:35.465619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9ddf8280-69fb-4c2d-9148-f5e26fcce57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:35.561635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 13:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:35.712617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea11b9-995b-40a4-8561-df4977593c0a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:35.714164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-faea11b9-995b-40a4-8561-df4977593c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:35.860303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea11b9-995b-40a4-8561-df4977593c0a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:35.872150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:35.974476 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91b1c5-8239-4e59-b081-a6e8da8c4413 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:35.975111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91b1c5-8239-4e59-b081-a6e8da8c4413 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:35.975392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91b1c5-8239-4e59-b081-a6e8da8c4413 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:35.975658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91b1c5-8239-4e59-b081-a6e8da8c4413 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:35.977275 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0b91b1c5-8239-4e59-b081-a6e8da8c4413 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:36.067314 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91b1c5-8239-4e59-b081-a6e8da8c4413 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:36.068915 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91b1c5-8239-4e59-b081-a6e8da8c4413 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:36.087222 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 706/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:37.201955 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38f88e-73f2-4d88-bb95-4e6a7346702c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:37.202504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38f88e-73f2-4d88-bb95-4e6a7346702c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:37.202706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38f88e-73f2-4d88-bb95-4e6a7346702c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:37.204190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 13:08:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:37.210467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efcf37-f956-48cc-b6ce-6533aeceb1eb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:37.210892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efcf37-f956-48cc-b6ce-6533aeceb1eb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:37.211003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efcf37-f956-48cc-b6ce-6533aeceb1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:37.231784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:37.288272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-11efcf37-f956-48cc-b6ce-6533aeceb1eb None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:37.289433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-11efcf37-f956-48cc-b6ce-6533aeceb1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:37.384798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 13:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:37.533545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38f88e-73f2-4d88-bb95-4e6a7346702c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:37.535134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9d38f88e-73f2-4d88-bb95-4e6a7346702c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:37.645295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:37.683391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38f88e-73f2-4d88-bb95-4e6a7346702c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:37.794813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d2ee8-5ee0-4ee1-b6f2-77f65415ce59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:37.796081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d2ee8-5ee0-4ee1-b6f2-77f65415ce59 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:37.796361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d2ee8-5ee0-4ee1-b6f2-77f65415ce59 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:37.796643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d2ee8-5ee0-4ee1-b6f2-77f65415ce59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:37.798231 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-703d2ee8-5ee0-4ee1-b6f2-77f65415ce59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:37.876423 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:37.890289 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d2ee8-5ee0-4ee1-b6f2-77f65415ce59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:37.891911 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d2ee8-5ee0-4ee1-b6f2-77f65415ce59 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:37.909694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 718/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:39.041897 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df828e-d0bc-46d9-9ec5-73f162aa3dfe None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:39.042432 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df828e-d0bc-46d9-9ec5-73f162aa3dfe None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:39.042559 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df828e-d0bc-46d9-9ec5-73f162aa3dfe None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:39.043846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 13:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:39.050696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfb97e-abfa-4ee8-a8d6-a0c349aa53dd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:39.051136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfb97e-abfa-4ee8-a8d6-a0c349aa53dd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:39.051310 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfb97e-abfa-4ee8-a8d6-a0c349aa53dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:39.126450 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-59dfb97e-abfa-4ee8-a8d6-a0c349aa53dd None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:39.127613 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-59dfb97e-abfa-4ee8-a8d6-a0c349aa53dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:39.222510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 13:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:39.373677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df828e-d0bc-46d9-9ec5-73f162aa3dfe None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:39.375978 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-95df828e-d0bc-46d9-9ec5-73f162aa3dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:39.459704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:39.521671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df828e-d0bc-46d9-9ec5-73f162aa3dfe None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:39.634984 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4459072-3571-48f4-8d35-fd27357a0651 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:39.635623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4459072-3571-48f4-8d35-fd27357a0651 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:39.635898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4459072-3571-48f4-8d35-fd27357a0651 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:39.636166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4459072-3571-48f4-8d35-fd27357a0651 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:39.637743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f4459072-3571-48f4-8d35-fd27357a0651 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:39.685331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:39.727481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4459072-3571-48f4-8d35-fd27357a0651 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:39.729030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4459072-3571-48f4-8d35-fd27357a0651 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:39.749505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 709/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:40.810201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ca370-a5c8-427e-9b27-db5371094138 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:40.810804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ca370-a5c8-427e-9b27-db5371094138 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:40.810804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ca370-a5c8-427e-9b27-db5371094138 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:40.812602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 13:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:40.818901 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b29e4-1180-4012-ba02-e2b27f3137bd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:40.819375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b29e4-1180-4012-ba02-e2b27f3137bd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:40.819550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b29e4-1180-4012-ba02-e2b27f3137bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:40.896075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-867b29e4-1180-4012-ba02-e2b27f3137bd None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:40.897295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-867b29e4-1180-4012-ba02-e2b27f3137bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:40.993660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 13:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:41.146072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ca370-a5c8-427e-9b27-db5371094138 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:41.147569 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-af4ca370-a5c8-427e-9b27-db5371094138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:41.196483 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:41.294446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ca370-a5c8-427e-9b27-db5371094138 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:41.405347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbc370-b786-48ff-8014-d149820a55ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:41.406542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbc370-b786-48ff-8014-d149820a55ec service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:41.406818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbc370-b786-48ff-8014-d149820a55ec service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:41.407119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbc370-b786-48ff-8014-d149820a55ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:41.408694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aebbc370-b786-48ff-8014-d149820a55ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:41.415546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:41.500836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbc370-b786-48ff-8014-d149820a55ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:41.502524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbc370-b786-48ff-8014-d149820a55ec service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:41.520431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 721/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:42.591998 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873e786-ba97-4e92-9d77-8dda8a0a0802 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:42.592528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873e786-ba97-4e92-9d77-8dda8a0a0802 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:42.592528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873e786-ba97-4e92-9d77-8dda8a0a0802 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:42.594486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 13:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:42.601344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2967edb-73a8-4a9d-8bdf-a658b23930c1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:42.601813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2967edb-73a8-4a9d-8bdf-a658b23930c1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:42.601924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2967edb-73a8-4a9d-8bdf-a658b23930c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:42.677491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a2967edb-73a8-4a9d-8bdf-a658b23930c1 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:42.678719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a2967edb-73a8-4a9d-8bdf-a658b23930c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:42.747382 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:42.777136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 13:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:42.928243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873e786-ba97-4e92-9d77-8dda8a0a0802 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:42.929774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c873e786-ba97-4e92-9d77-8dda8a0a0802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:42.947978 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:43.077427 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873e786-ba97-4e92-9d77-8dda8a0a0802 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:43.164177 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:43.191225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc3507-e0b3-4c57-8f50-af373ab21909 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:43.191771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc3507-e0b3-4c57-8f50-af373ab21909 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:43.192049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc3507-e0b3-4c57-8f50-af373ab21909 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:43.192321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc3507-e0b3-4c57-8f50-af373ab21909 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:43.193914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-decc3507-e0b3-4c57-8f50-af373ab21909 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:43.282120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc3507-e0b3-4c57-8f50-af373ab21909 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:43.283778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc3507-e0b3-4c57-8f50-af373ab21909 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:43.302673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 712/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:44.369307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e139aa9-f7d0-4c0f-b1b8-ee4aa45fe5df None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:44.369845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e139aa9-f7d0-4c0f-b1b8-ee4aa45fe5df None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:44.369845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e139aa9-f7d0-4c0f-b1b8-ee4aa45fe5df None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:44.371139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 13:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:44.377297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37dd82-5bf6-4b0b-b9ae-7e095e167796 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:44.377708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37dd82-5bf6-4b0b-b9ae-7e095e167796 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:44.377824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37dd82-5bf6-4b0b-b9ae-7e095e167796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:44.453922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-db37dd82-5bf6-4b0b-b9ae-7e095e167796 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:44.455163 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-db37dd82-5bf6-4b0b-b9ae-7e095e167796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:44.497040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:44.552071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 13:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:44.706821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e139aa9-f7d0-4c0f-b1b8-ee4aa45fe5df None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:44.708367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9e139aa9-f7d0-4c0f-b1b8-ee4aa45fe5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:44.852612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e139aa9-f7d0-4c0f-b1b8-ee4aa45fe5df None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:44.905767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:44.970372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe5e0c-2f74-4133-820b-88d316ac8e7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:44.970905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe5e0c-2f74-4133-820b-88d316ac8e7b service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:44.971219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe5e0c-2f74-4133-820b-88d316ac8e7b service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:44.971493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe5e0c-2f74-4133-820b-88d316ac8e7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:44.973079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-adfe5e0c-2f74-4133-820b-88d316ac8e7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:45.067570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe5e0c-2f74-4133-820b-88d316ac8e7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:45.069192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe5e0c-2f74-4133-820b-88d316ac8e7b service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:45.087815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 724/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:46.147948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61f3f4-12e4-4e77-8f18-baadf4b00d53 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:46.148521 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61f3f4-12e4-4e77-8f18-baadf4b00d53 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:46.148521 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61f3f4-12e4-4e77-8f18-baadf4b00d53 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:46.149861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 13:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:46.156205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51069229-6d3a-422b-8f31-d42766050599 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:46.156638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51069229-6d3a-422b-8f31-d42766050599 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:46.156737 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51069229-6d3a-422b-8f31-d42766050599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:46.231933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-51069229-6d3a-422b-8f31-d42766050599 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:46.233155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-51069229-6d3a-422b-8f31-d42766050599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:46.235238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:46.330795 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 13:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:46.481576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61f3f4-12e4-4e77-8f18-baadf4b00d53 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:46.483099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6e61f3f4-12e4-4e77-8f18-baadf4b00d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:46.628556 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61f3f4-12e4-4e77-8f18-baadf4b00d53 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:46.656132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:46.742432 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206ad95-54fb-4ede-80be-399eedb97c42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:46.742984 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206ad95-54fb-4ede-80be-399eedb97c42 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:46.743297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206ad95-54fb-4ede-80be-399eedb97c42 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:46.743561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206ad95-54fb-4ede-80be-399eedb97c42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:46.745138 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f206ad95-54fb-4ede-80be-399eedb97c42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:46.834138 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206ad95-54fb-4ede-80be-399eedb97c42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:46.835787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206ad95-54fb-4ede-80be-399eedb97c42 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:46.853733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 715/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:48.936501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76001fe7-d70d-46f5-babf-364e8c2f9b99 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:48.937045 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76001fe7-d70d-46f5-babf-364e8c2f9b99 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:48.937045 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76001fe7-d70d-46f5-babf-364e8c2f9b99 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:48.938658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:48.944627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c13aab-4c11-4db8-9333-13f67f175805 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:48.945051 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c13aab-4c11-4db8-9333-13f67f175805 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:48.945164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c13aab-4c11-4db8-9333-13f67f175805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:48.985898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:49.011715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b4c13aab-4c11-4db8-9333-13f67f175805 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:49.012928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b4c13aab-4c11-4db8-9333-13f67f175805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:49.110612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:49.395078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76001fe7-d70d-46f5-babf-364e8c2f9b99 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:49.396641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-76001fe7-d70d-46f5-babf-364e8c2f9b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:49.502502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76001fe7-d70d-46f5-babf-364e8c2f9b99 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:49.535159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:49.578527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30864f7f-cc05-40a5-aa84-6e589c798bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:49.579164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30864f7f-cc05-40a5-aa84-6e589c798bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:49.579448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30864f7f-cc05-40a5-aa84-6e589c798bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:49.579722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30864f7f-cc05-40a5-aa84-6e589c798bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:49.581341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-30864f7f-cc05-40a5-aa84-6e589c798bb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:49.692838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30864f7f-cc05-40a5-aa84-6e589c798bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:49.694306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30864f7f-cc05-40a5-aa84-6e589c798bb4 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:49.712757 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 727/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:52.029231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8448f92-aed5-446d-81a0-6a46405a4baf None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:52.029809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8448f92-aed5-446d-81a0-6a46405a4baf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:52.029809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8448f92-aed5-446d-81a0-6a46405a4baf None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:52.031210 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:52.036550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98fe3e-c55b-4567-8ef3-2d4e0142f2ba None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:52.036936 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98fe3e-c55b-4567-8ef3-2d4e0142f2ba None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:52.037106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98fe3e-c55b-4567-8ef3-2d4e0142f2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:52.102873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-be98fe3e-c55b-4567-8ef3-2d4e0142f2ba None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:52.104130 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-be98fe3e-c55b-4567-8ef3-2d4e0142f2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:52.117365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:52.203006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:52.491072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8448f92-aed5-446d-81a0-6a46405a4baf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:52.492665 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e8448f92-aed5-446d-81a0-6a46405a4baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:52.596015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8448f92-aed5-446d-81a0-6a46405a4baf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:52.609321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:52.670588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff31796-1e48-44c9-a12c-e04991977992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:52.671229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff31796-1e48-44c9-a12c-e04991977992 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:52.671509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff31796-1e48-44c9-a12c-e04991977992 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:52.671783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff31796-1e48-44c9-a12c-e04991977992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:52.673377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fff31796-1e48-44c9-a12c-e04991977992 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:52.783349 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff31796-1e48-44c9-a12c-e04991977992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:52.784740 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff31796-1e48-44c9-a12c-e04991977992 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:52.802404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 718/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:55.264817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0daa93-14ce-4348-9634-8198931face6 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:55.265406 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0daa93-14ce-4348-9634-8198931face6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:55.265406 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0daa93-14ce-4348-9634-8198931face6 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:55.266720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:55.272692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1767617-2492-457d-92be-b050990432d8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:55.273107 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1767617-2492-457d-92be-b050990432d8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:55.273246 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1767617-2492-457d-92be-b050990432d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:55.321905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:55.340268 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e1767617-2492-457d-92be-b050990432d8 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:55.341504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e1767617-2492-457d-92be-b050990432d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:55.439975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:55.729571 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0daa93-14ce-4348-9634-8198931face6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:55.731136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4c0daa93-14ce-4348-9634-8198931face6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:55.835537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0daa93-14ce-4348-9634-8198931face6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:55.875159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:55.909176 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06e2f1-d76f-4126-b297-4592d8efccd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:55.909781 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06e2f1-d76f-4126-b297-4592d8efccd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:55.910060 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06e2f1-d76f-4126-b297-4592d8efccd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:55.910353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06e2f1-d76f-4126-b297-4592d8efccd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:55.912243 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9d06e2f1-d76f-4126-b297-4592d8efccd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:56.022137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06e2f1-d76f-4126-b297-4592d8efccd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:56.023595 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06e2f1-d76f-4126-b297-4592d8efccd9 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:56.041520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 730/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:58.390027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afbea5-352e-43a5-8b9b-a32e0b570460 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:58.390608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afbea5-352e-43a5-8b9b-a32e0b570460 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:58.390608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afbea5-352e-43a5-8b9b-a32e0b570460 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:58.391955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:08:58.397935 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb992ca-3b92-4f81-b79e-311aa82b470c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:58.398357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb992ca-3b92-4f81-b79e-311aa82b470c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:58.398464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb992ca-3b92-4f81-b79e-311aa82b470c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:58.467868 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4eb992ca-3b92-4f81-b79e-311aa82b470c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:58.469354 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4eb992ca-3b92-4f81-b79e-311aa82b470c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:58.492765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:58.570052 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:58.862069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afbea5-352e-43a5-8b9b-a32e0b570460 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:58.863916 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-25afbea5-352e-43a5-8b9b-a32e0b570460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:58.978022 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afbea5-352e-43a5-8b9b-a32e0b570460 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:58.999010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:59.054194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3890087-4806-4530-9793-df4ea0f0727c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:59.054809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3890087-4806-4530-9793-df4ea0f0727c service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:59.055114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3890087-4806-4530-9793-df4ea0f0727c service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:59.055392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3890087-4806-4530-9793-df4ea0f0727c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:59.056991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c3890087-4806-4530-9793-df4ea0f0727c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:59.174859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3890087-4806-4530-9793-df4ea0f0727c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:59.176336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3890087-4806-4530-9793-df4ea0f0727c service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:59.195994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 721/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:01.493322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021a8f5-057c-4e16-abd2-dde1d37b5e33 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:01.493864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021a8f5-057c-4e16-abd2-dde1d37b5e33 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:01.493864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021a8f5-057c-4e16-abd2-dde1d37b5e33 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:01.495264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:01.500824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee49e4-7e0f-4303-9305-31490a4eb79f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:01.501265 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee49e4-7e0f-4303-9305-31490a4eb79f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:01.501365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee49e4-7e0f-4303-9305-31490a4eb79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:01.564791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:01.567936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9eee49e4-7e0f-4303-9305-31490a4eb79f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:01.569832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9eee49e4-7e0f-4303-9305-31490a4eb79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:01.666884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:01.953227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021a8f5-057c-4e16-abd2-dde1d37b5e33 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:01.954836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e021a8f5-057c-4e16-abd2-dde1d37b5e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:02.067855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021a8f5-057c-4e16-abd2-dde1d37b5e33 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:02.117264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:02.141129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb948ca8-a3f9-4311-933a-a6433021d680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:02.141735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948ca8-a3f9-4311-933a-a6433021d680 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:02.142013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948ca8-a3f9-4311-933a-a6433021d680 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:02.142286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948ca8-a3f9-4311-933a-a6433021d680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:02.143898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cb948ca8-a3f9-4311-933a-a6433021d680 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:02.369962 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb948ca8-a3f9-4311-933a-a6433021d680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:02.371409 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb948ca8-a3f9-4311-933a-a6433021d680 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:02.389550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 733/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:04.600076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca844d1e-f889-4e83-b4b4-0fa4a84f3b4a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:04.601575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca844d1e-f889-4e83-b4b4-0fa4a84f3b4a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:04.601575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca844d1e-f889-4e83-b4b4-0fa4a84f3b4a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:04.602419 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:09:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:04.607982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e1681-c30c-41ee-949b-e1a1d8676f1c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:04.608442 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e1681-c30c-41ee-949b-e1a1d8676f1c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:04.608559 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e1681-c30c-41ee-949b-e1a1d8676f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:04.673931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-158e1681-c30c-41ee-949b-e1a1d8676f1c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:04.675155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-158e1681-c30c-41ee-949b-e1a1d8676f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:04.709012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:04.777779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:05.070016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca844d1e-f889-4e83-b4b4-0fa4a84f3b4a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:05.071914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ca844d1e-f889-4e83-b4b4-0fa4a84f3b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:05.174544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca844d1e-f889-4e83-b4b4-0fa4a84f3b4a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:05.204347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:05.249963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b55e2-125c-4e82-af96-ba65dd923f74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:05.250565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b55e2-125c-4e82-af96-ba65dd923f74 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:05.250839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b55e2-125c-4e82-af96-ba65dd923f74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:05.251144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b55e2-125c-4e82-af96-ba65dd923f74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:05.252717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ad0b55e2-125c-4e82-af96-ba65dd923f74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:05.365304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b55e2-125c-4e82-af96-ba65dd923f74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:05.367308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b55e2-125c-4e82-af96-ba65dd923f74 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:05.385510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 724/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:07.584141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1479c6b-ae83-4346-9694-243a7c29d650 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:07.584713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1479c6b-ae83-4346-9694-243a7c29d650 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:07.584713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1479c6b-ae83-4346-9694-243a7c29d650 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:07.586455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:07.591967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56818eb7-c19c-4c3b-bc7f-c0356914ba72 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:07.592407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56818eb7-c19c-4c3b-bc7f-c0356914ba72 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:07.592506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56818eb7-c19c-4c3b-bc7f-c0356914ba72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:07.658604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-56818eb7-c19c-4c3b-bc7f-c0356914ba72 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:07.659854 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-56818eb7-c19c-4c3b-bc7f-c0356914ba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:07.670098 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:07.763015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:08.050481 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1479c6b-ae83-4346-9694-243a7c29d650 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:08.052070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e1479c6b-ae83-4346-9694-243a7c29d650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:08.157290 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1479c6b-ae83-4346-9694-243a7c29d650 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:08.213236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:08.232133 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914f568-b807-496e-9b67-4fff702608e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:08.232635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914f568-b807-496e-9b67-4fff702608e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:08.232895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914f568-b807-496e-9b67-4fff702608e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:08.233165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914f568-b807-496e-9b67-4fff702608e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:08.234768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f914f568-b807-496e-9b67-4fff702608e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:08.344595 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914f568-b807-496e-9b67-4fff702608e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:08.345954 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914f568-b807-496e-9b67-4fff702608e3 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:08.363762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 736/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:10.564698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb8446-4011-498a-9ad6-4c21acb5625f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:10.565343 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb8446-4011-498a-9ad6-4c21acb5625f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:10.565343 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb8446-4011-498a-9ad6-4c21acb5625f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:10.566622 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:10.572920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e59d74-1b10-47c1-b91f-06a237a203de None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:10.573361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e59d74-1b10-47c1-b91f-06a237a203de None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:10.573472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e59d74-1b10-47c1-b91f-06a237a203de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:10.639660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c3e59d74-1b10-47c1-b91f-06a237a203de None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:10.640858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c3e59d74-1b10-47c1-b91f-06a237a203de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:10.678613 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:10.736927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:11.022990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb8446-4011-498a-9ad6-4c21acb5625f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:11.024589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3ceb8446-4011-498a-9ad6-4c21acb5625f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:11.127594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb8446-4011-498a-9ad6-4c21acb5625f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:11.165370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:11.202685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f4a38-ce42-417a-9674-68a1ee79bc62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:11.203424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f4a38-ce42-417a-9674-68a1ee79bc62 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:11.203723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f4a38-ce42-417a-9674-68a1ee79bc62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:11.204004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f4a38-ce42-417a-9674-68a1ee79bc62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:11.206758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-806f4a38-ce42-417a-9674-68a1ee79bc62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:11.324203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f4a38-ce42-417a-9674-68a1ee79bc62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:11.325581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f4a38-ce42-417a-9674-68a1ee79bc62 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:11.343951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 727/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:13.497130 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff3ed6-d0df-4b39-853e-7457f18eff31 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:13.497703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff3ed6-d0df-4b39-853e-7457f18eff31 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:13.497703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff3ed6-d0df-4b39-853e-7457f18eff31 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:13.499016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:13.504420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb319876-b485-4cbd-9629-ef7d15ac1c68 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:13.504835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb319876-b485-4cbd-9629-ef7d15ac1c68 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:13.504970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb319876-b485-4cbd-9629-ef7d15ac1c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:13.569884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cb319876-b485-4cbd-9629-ef7d15ac1c68 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:13.571382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cb319876-b485-4cbd-9629-ef7d15ac1c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:13.588955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:13.667952 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:13.951336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff3ed6-d0df-4b39-853e-7457f18eff31 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:13.952869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9fff3ed6-d0df-4b39-853e-7457f18eff31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:14.058320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff3ed6-d0df-4b39-853e-7457f18eff31 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:14.125555 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:14.132764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1f3bd-151e-43dc-8557-2176208eef3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:14.133367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1f3bd-151e-43dc-8557-2176208eef3b service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:14.133640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1f3bd-151e-43dc-8557-2176208eef3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:14.133908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1f3bd-151e-43dc-8557-2176208eef3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:14.135647 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-65c1f3bd-151e-43dc-8557-2176208eef3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:14.243256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1f3bd-151e-43dc-8557-2176208eef3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:14.245365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1f3bd-151e-43dc-8557-2176208eef3b service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:14.263768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 739/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:16.441847 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690890d8-6c62-4ca6-885e-876f37ee5b54 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:16.442431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690890d8-6c62-4ca6-885e-876f37ee5b54 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:16.442431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690890d8-6c62-4ca6-885e-876f37ee5b54 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:16.443797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:16.450282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1b3ef-2ef5-4e9e-aa86-75dd3d15a3a4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:16.450715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1b3ef-2ef5-4e9e-aa86-75dd3d15a3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:16.450828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1b3ef-2ef5-4e9e-aa86-75dd3d15a3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:16.515520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ebc1b3ef-2ef5-4e9e-aa86-75dd3d15a3a4 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:16.516639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ebc1b3ef-2ef5-4e9e-aa86-75dd3d15a3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:16.565929 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:16.615528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:16.898820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690890d8-6c62-4ca6-885e-876f37ee5b54 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:16.900398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-690890d8-6c62-4ca6-885e-876f37ee5b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:17.004262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690890d8-6c62-4ca6-885e-876f37ee5b54 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:17.052436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:17.078116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd00a3c-f060-4d0b-b23f-3a111d3654c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:17.078717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd00a3c-f060-4d0b-b23f-3a111d3654c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:17.078995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd00a3c-f060-4d0b-b23f-3a111d3654c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:17.079293 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd00a3c-f060-4d0b-b23f-3a111d3654c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:17.080932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-afd00a3c-f060-4d0b-b23f-3a111d3654c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:17.189991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd00a3c-f060-4d0b-b23f-3a111d3654c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:17.191485 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd00a3c-f060-4d0b-b23f-3a111d3654c2 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:17.210475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 730/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:26.613958 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=22 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:27.208204 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=66 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:37.239720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31b908-4de3-4a23-9d39-f7a0792031f2 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:37.240287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31b908-4de3-4a23-9d39-f7a0792031f2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:37.240287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31b908-4de3-4a23-9d39-f7a0792031f2 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:37.241639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 741/1471] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:37.246895 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8cc01-12eb-46b0-b3dd-1f1fc250fa6b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:37.247296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8cc01-12eb-46b0-b3dd-1f1fc250fa6b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:37.247467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8cc01-12eb-46b0-b3dd-1f1fc250fa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:37.312255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-86d8cc01-12eb-46b0-b3dd-1f1fc250fa6b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:37.313380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-86d8cc01-12eb-46b0-b3dd-1f1fc250fa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:37.414566 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 731/1472] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:37.718920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574df77-0887-40a6-a74c-317c2e239ac9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:37.719268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574df77-0887-40a6-a74c-317c2e239ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:37.719543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574df77-0887-40a6-a74c-317c2e239ac9 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:37.720877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 742/1473] 10.222.0.17 () {54 vars in 966 bytes} [Fri Sep 9 13:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:37.725983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50e2cc-05c8-41db-b279-56e207878f98 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:37.726264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50e2cc-05c8-41db-b279-56e207878f98 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:37.726533 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50e2cc-05c8-41db-b279-56e207878f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:37.802336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2c50e2cc-05c8-41db-b279-56e207878f98 None None] MFA Rules not processed for user `a7ef6cd64182474787134e3227beb3aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:37.803574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2c50e2cc-05c8-41db-b279-56e207878f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:37.824588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:37.880240 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 732/1474] 10.222.0.17 () {58 vars in 1063 bytes} [Fri Sep 9 13:09:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:37.887876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574df77-0887-40a6-a74c-317c2e239ac9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:37.889386 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c574df77-0887-40a6-a74c-317c2e239ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:37.991342 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574df77-0887-40a6-a74c-317c2e239ac9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:38.064609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a24ef5-e632-416d-9c81-b36a936dae43 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ef6cd64182474787134e3227beb3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:38.065216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a24ef5-e632-416d-9c81-b36a936dae43 service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:38.065493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a24ef5-e632-416d-9c81-b36a936dae43 service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:38.065761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a24ef5-e632-416d-9c81-b36a936dae43 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:38.067361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-25a24ef5-e632-416d-9c81-b36a936dae43 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:38.121879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:38.177447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a24ef5-e632-416d-9c81-b36a936dae43 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:38.178880 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a24ef5-e632-416d-9c81-b36a936dae43 service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:38.196898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 743/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:40.442181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21629204-0610-467d-bb84-9ad5d42b0271 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:40.442662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21629204-0610-467d-bb84-9ad5d42b0271 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:40.442759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21629204-0610-467d-bb84-9ad5d42b0271 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:40.444139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 733/1476] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:40.449470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46352466-49c1-4d8c-bf6a-0c90b62534c2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:40.449932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46352466-49c1-4d8c-bf6a-0c90b62534c2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:40.450081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46352466-49c1-4d8c-bf6a-0c90b62534c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:40.517228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-46352466-49c1-4d8c-bf6a-0c90b62534c2 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:40.518462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-46352466-49c1-4d8c-bf6a-0c90b62534c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:40.619191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 744/1477] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:09:40.896508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21629204-0610-467d-bb84-9ad5d42b0271 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:40.897955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-21629204-0610-467d-bb84-9ad5d42b0271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:41.024535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21629204-0610-467d-bb84-9ad5d42b0271 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:41.073556 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:41.117890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2afa4-5955-4edf-b953-53d8449fd2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:41.118491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2afa4-5955-4edf-b953-53d8449fd2db admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:41.118764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2afa4-5955-4edf-b953-53d8449fd2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:41.119027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2afa4-5955-4edf-b953-53d8449fd2db admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:41.119685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2afa4-5955-4edf-b953-53d8449fd2db admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:41.121451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-7ae2afa4-5955-4edf-b953-53d8449fd2db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:09:41.121451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 09 13:09:41.121451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: Sep 09 13:09:41.122589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2afa4-5955-4edf-b953-53d8449fd2db admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:41.142989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 734/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 9 13:09:40 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:09:43.306808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73552562-9fe8-4eab-9549-c112f261e5c3 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:43.307488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73552562-9fe8-4eab-9549-c112f261e5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:43.307488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73552562-9fe8-4eab-9549-c112f261e5c3 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:43.308759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 745/1479] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:43.313961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bdfbf-cea0-4cec-94be-d637518c7c04 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:43.314375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bdfbf-cea0-4cec-94be-d637518c7c04 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:43.314504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bdfbf-cea0-4cec-94be-d637518c7c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:43.381012 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-821bdfbf-cea0-4cec-94be-d637518c7c04 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:43.382243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-821bdfbf-cea0-4cec-94be-d637518c7c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:43.480107 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 735/1480] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:43.771118 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73552562-9fe8-4eab-9549-c112f261e5c3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:43.772695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-73552562-9fe8-4eab-9549-c112f261e5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:43.783330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:43.917363 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73552562-9fe8-4eab-9549-c112f261e5c3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:43.996929 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:44.034748 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc6316-5a9d-4802-ac17-be3060c116c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:44.035385 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc6316-5a9d-4802-ac17-be3060c116c0 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:44.035657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc6316-5a9d-4802-ac17-be3060c116c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:44.035922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc6316-5a9d-4802-ac17-be3060c116c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:44.037508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-acfc6316-5a9d-4802-ac17-be3060c116c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:44.127622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc6316-5a9d-4802-ac17-be3060c116c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:44.129216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc6316-5a9d-4802-ac17-be3060c116c0 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:44.147883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 746/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:46.472982 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702679b-fd0e-4f0f-babb-09ec5537a550 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:46.473544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702679b-fd0e-4f0f-babb-09ec5537a550 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:46.473544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702679b-fd0e-4f0f-babb-09ec5537a550 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:46.474917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 736/1482] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:46.480305 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e314b73-bee0-4f1e-a831-59733ea75717 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:46.480727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e314b73-bee0-4f1e-a831-59733ea75717 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:46.480836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e314b73-bee0-4f1e-a831-59733ea75717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:46.546813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5e314b73-bee0-4f1e-a831-59733ea75717 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:46.548056 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5e314b73-bee0-4f1e-a831-59733ea75717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:46.594198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:46.645428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 747/1483] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:46.937626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702679b-fd0e-4f0f-babb-09ec5537a550 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:46.939716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b702679b-fd0e-4f0f-babb-09ec5537a550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:46.964042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:47.048821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702679b-fd0e-4f0f-babb-09ec5537a550 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:47.121285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb32b0d-5186-496b-ad9e-d4b2c80b2ff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:47.121920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb32b0d-5186-496b-ad9e-d4b2c80b2ff3 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:47.122196 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb32b0d-5186-496b-ad9e-d4b2c80b2ff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:47.122466 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb32b0d-5186-496b-ad9e-d4b2c80b2ff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:47.124245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0cb32b0d-5186-496b-ad9e-d4b2c80b2ff3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:47.199647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:47.236992 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb32b0d-5186-496b-ad9e-d4b2c80b2ff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:47.238466 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb32b0d-5186-496b-ad9e-d4b2c80b2ff3 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:47.256544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 737/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:49.435888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831f413-9c29-43ed-b304-7238d011c281 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:49.436470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831f413-9c29-43ed-b304-7238d011c281 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:49.436470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831f413-9c29-43ed-b304-7238d011c281 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:49.437766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 748/1485] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:49.443710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129227fe-f9f6-419c-b8a0-d991dad74107 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:49.444118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129227fe-f9f6-419c-b8a0-d991dad74107 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:49.444278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129227fe-f9f6-419c-b8a0-d991dad74107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:49.511845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-129227fe-f9f6-419c-b8a0-d991dad74107 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:49.513080 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-129227fe-f9f6-419c-b8a0-d991dad74107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:49.611365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 738/1486] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:49.899286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831f413-9c29-43ed-b304-7238d011c281 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:49.900861 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b831f413-9c29-43ed-b304-7238d011c281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:50.004812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831f413-9c29-43ed-b304-7238d011c281 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:50.076516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:50.077986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a3077-f8fd-4d18-86e3-048e3e93d9ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:50.078601 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a3077-f8fd-4d18-86e3-048e3e93d9ba service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:50.079092 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a3077-f8fd-4d18-86e3-048e3e93d9ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:50.079529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a3077-f8fd-4d18-86e3-048e3e93d9ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:50.081670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3a9a3077-f8fd-4d18-86e3-048e3e93d9ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:50.190182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a3077-f8fd-4d18-86e3-048e3e93d9ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:50.191645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a3077-f8fd-4d18-86e3-048e3e93d9ba service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:50.210354 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 749/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:52.449681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d852938-746a-4e37-ad9f-e0ca8f42febe None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:52.450242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d852938-746a-4e37-ad9f-e0ca8f42febe None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:52.450242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d852938-746a-4e37-ad9f-e0ca8f42febe None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:52.451606 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 739/1488] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:52.457389 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31b836-e1ea-477c-b09f-3db9bdd193e9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:52.457818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31b836-e1ea-477c-b09f-3db9bdd193e9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:52.457923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31b836-e1ea-477c-b09f-3db9bdd193e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:52.522185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7b31b836-e1ea-477c-b09f-3db9bdd193e9 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:52.523339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7b31b836-e1ea-477c-b09f-3db9bdd193e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:52.575814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:52.621134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 750/1489] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:52.916024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d852938-746a-4e37-ad9f-e0ca8f42febe None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:52.917582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5d852938-746a-4e37-ad9f-e0ca8f42febe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:52.953425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:53.025389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d852938-746a-4e37-ad9f-e0ca8f42febe None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:53.098921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99355c-04df-4ea4-95ad-b1bb8d2989a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:53.099549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99355c-04df-4ea4-95ad-b1bb8d2989a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:53.099824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99355c-04df-4ea4-95ad-b1bb8d2989a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:53.100092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99355c-04df-4ea4-95ad-b1bb8d2989a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:53.101686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cc99355c-04df-4ea4-95ad-b1bb8d2989a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:53.300571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:53.331234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99355c-04df-4ea4-95ad-b1bb8d2989a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:53.332671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99355c-04df-4ea4-95ad-b1bb8d2989a9 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:53.350272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 740/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:55.708431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee038d5-4b87-4bf8-b7c1-584b34c522d2 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:55.708991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee038d5-4b87-4bf8-b7c1-584b34c522d2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:55.708991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee038d5-4b87-4bf8-b7c1-584b34c522d2 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:55.710338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 751/1491] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 9 13:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:09:55.715743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b886-bef9-45b2-b126-17921094126e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:55.716172 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b886-bef9-45b2-b126-17921094126e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:55.716287 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b886-bef9-45b2-b126-17921094126e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:55.782066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d076b886-bef9-45b2-b126-17921094126e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:55.783315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d076b886-bef9-45b2-b126-17921094126e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:55.879003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 741/1492] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 9 13:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:56.172229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee038d5-4b87-4bf8-b7c1-584b34c522d2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:56.173763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2ee038d5-4b87-4bf8-b7c1-584b34c522d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:56.276659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee038d5-4b87-4bf8-b7c1-584b34c522d2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:56.350486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89164ce7-158e-4c57-95b6-11f2254ffabd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:56.351109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89164ce7-158e-4c57-95b6-11f2254ffabd service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:56.351390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89164ce7-158e-4c57-95b6-11f2254ffabd service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:56.351661 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89164ce7-158e-4c57-95b6-11f2254ffabd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:56.353241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-89164ce7-158e-4c57-95b6-11f2254ffabd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:56.361660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:56.462645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89164ce7-158e-4c57-95b6-11f2254ffabd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:56.464106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89164ce7-158e-4c57-95b6-11f2254ffabd service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:56.482123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 752/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:10:05.877477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=82 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:10:06.479871 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=50 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:09.310668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27984748-60be-43b2-a4ec-b1513d1c275b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:09.311288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27984748-60be-43b2-a4ec-b1513d1c275b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:09.311288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27984748-60be-43b2-a4ec-b1513d1c275b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:09.387162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-27984748-60be-43b2-a4ec-b1513d1c275b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:11:09.388557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-27984748-60be-43b2-a4ec-b1513d1c275b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:09.484418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 742/1494] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:11:09.492302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64d52e-60a0-4721-af2f-66a14f757c30 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:09.492792 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64d52e-60a0-4721-af2f-66a14f757c30 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:09.492901 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64d52e-60a0-4721-af2f-66a14f757c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:09.568890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7a64d52e-60a0-4721-af2f-66a14f757c30 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:11:09.570027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7a64d52e-60a0-4721-af2f-66a14f757c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:09.665483 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 753/1495] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:11:09.670888 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27984748-60be-43b2-a4ec-b1513d1c275b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:11:09.672150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-27984748-60be-43b2-a4ec-b1513d1c275b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:09.683308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27984748-60be-43b2-a4ec-b1513d1c275b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:11:09.746791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:09.782315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b4efb-85ef-4abf-9b45-37587a5c7991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:11:09.782926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b4efb-85ef-4abf-9b45-37587a5c7991 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:09.783243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b4efb-85ef-4abf-9b45-37587a5c7991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:09.783509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b4efb-85ef-4abf-9b45-37587a5c7991 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:09.784142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b4efb-85ef-4abf-9b45-37587a5c7991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:11:09.784442 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b4efb-85ef-4abf-9b45-37587a5c7991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:11:09.785453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-ce3b4efb-85ef-4abf-9b45-37587a5c7991 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:11:09.785453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 09 13:11:09.785453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: Sep 09 13:11:09.786331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b4efb-85ef-4abf-9b45-37587a5c7991 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:11:09.798922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 743/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:11:09.807275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64d52e-60a0-4721-af2f-66a14f757c30 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:11:09.816408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64d52e-60a0-4721-af2f-66a14f757c30 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:11:09.877742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:09.911614 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580fcd0-e814-474e-9d3d-d731acea5641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:11:09.912219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580fcd0-e814-474e-9d3d-d731acea5641 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:09.912497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580fcd0-e814-474e-9d3d-d731acea5641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:09.912767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580fcd0-e814-474e-9d3d-d731acea5641 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:09.913401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580fcd0-e814-474e-9d3d-d731acea5641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:11:09.915151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-d580fcd0-e814-474e-9d3d-d731acea5641 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:11:09.915151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Sep 09 13:11:09.915151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: Sep 09 13:11:09.916356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580fcd0-e814-474e-9d3d-d731acea5641 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:11:09.972544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 754/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:11:09 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:11:09.976705 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b4efb-85ef-4abf-9b45-37587a5c7991 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:11:09.978247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ce3b4efb-85ef-4abf-9b45-37587a5c7991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:10.011705 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b4efb-85ef-4abf-9b45-37587a5c7991 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:11:10.104987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960905f-7acb-4fde-9abe-b9eadecdbfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:11:10.105596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960905f-7acb-4fde-9abe-b9eadecdbfb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:10.105875 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960905f-7acb-4fde-9abe-b9eadecdbfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:10.106146 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960905f-7acb-4fde-9abe-b9eadecdbfb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:10.107649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960905f-7acb-4fde-9abe-b9eadecdbfb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:11:10.108672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-9960905f-7acb-4fde-9abe-b9eadecdbfb9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:11:10.108672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 09 13:11:10.108672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: Sep 09 13:11:10.109882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960905f-7acb-4fde-9abe-b9eadecdbfb9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:11:10.155509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 744/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:11:09 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:11:10.159508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580fcd0-e814-474e-9d3d-d731acea5641 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:11:10.161018 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d580fcd0-e814-474e-9d3d-d731acea5641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:10.223917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580fcd0-e814-474e-9d3d-d731acea5641 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:11:10.316837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:10.328855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8c723-4c1c-44f5-a03d-a6e0ad0fc2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:11:10.329464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8c723-4c1c-44f5-a03d-a6e0ad0fc2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:10.329744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8c723-4c1c-44f5-a03d-a6e0ad0fc2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:10.330021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8c723-4c1c-44f5-a03d-a6e0ad0fc2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:10.330665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8c723-4c1c-44f5-a03d-a6e0ad0fc2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:11:10.331718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-c3f8c723-4c1c-44f5-a03d-a6e0ad0fc2e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:11:10.331718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Sep 09 13:11:10.331718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: Sep 09 13:11:10.332617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8c723-4c1c-44f5-a03d-a6e0ad0fc2e0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:11:10.345749 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 755/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:11:10.349743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960905f-7acb-4fde-9abe-b9eadecdbfb9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:11:10.351271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9960905f-7acb-4fde-9abe-b9eadecdbfb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:10.362366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960905f-7acb-4fde-9abe-b9eadecdbfb9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:11:10.386663 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:10.460909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540661fb-d2df-45f3-9c77-c7d0f9222baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:11:10.461524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540661fb-d2df-45f3-9c77-c7d0f9222baf admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:10.461799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540661fb-d2df-45f3-9c77-c7d0f9222baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:10.462063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540661fb-d2df-45f3-9c77-c7d0f9222baf admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:10.462706 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540661fb-d2df-45f3-9c77-c7d0f9222baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:11:10.463796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-540661fb-d2df-45f3-9c77-c7d0f9222baf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:11:10.463796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 09 13:11:10.463796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: Sep 09 13:11:10.464678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540661fb-d2df-45f3-9c77-c7d0f9222baf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:11:10.477586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 745/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:11:10.482096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8c723-4c1c-44f5-a03d-a6e0ad0fc2e0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:11:10.483770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c3f8c723-4c1c-44f5-a03d-a6e0ad0fc2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:10.520074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8c723-4c1c-44f5-a03d-a6e0ad0fc2e0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:11:10.616167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9900d8-7555-4a56-8f9f-fb69712155cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:11:10.616830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9900d8-7555-4a56-8f9f-fb69712155cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:10.617106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9900d8-7555-4a56-8f9f-fb69712155cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:10.617375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9900d8-7555-4a56-8f9f-fb69712155cb admin admin] PATH_INFO: `/v3/projects/fe5a65456d5b487bb7d478e66cb30833/users/213d942b9f164d9eab0c2631f1666b0f/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:10.641247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9900d8-7555-4a56-8f9f-fb69712155cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5a65456d5b487bb7d478e66cb30833, user_id=213d942b9f164d9eab0c2631f1666b0f, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:11:10.642304 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-ac9900d8-7555-4a56-8f9f-fb69712155cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:11:10.642304 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Sep 09 13:11:10.642304 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: Sep 09 13:11:10.644279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9900d8-7555-4a56-8f9f-fb69712155cb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:11:10.667770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 756/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:11:10 2022] PUT /identity/v3/projects/fe5a65456d5b487bb7d478e66cb30833/users/213d942b9f164d9eab0c2631f1666b0f/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:11:10.680143 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cece15-2682-4de2-a312-ecfd4b741334 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:10.680438 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cece15-2682-4de2-a312-ecfd4b741334 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:10.680739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cece15-2682-4de2-a312-ecfd4b741334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:10.738267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-19cece15-2682-4de2-a312-ecfd4b741334 None None] MFA Rules not processed for user `213d942b9f164d9eab0c2631f1666b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:11:10.739519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-19cece15-2682-4de2-a312-ecfd4b741334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:10.816404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 746/1502] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:11:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:11:10.830376 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f17a8-1996-4cd2-afad-c54b8fe50616 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:10.830666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f17a8-1996-4cd2-afad-c54b8fe50616 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:10.830942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f17a8-1996-4cd2-afad-c54b8fe50616 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:10.832331 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 757/1503] 10.222.0.17 () {54 vars in 964 bytes} [Fri Sep 9 13:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:11:10.837155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad81c2-9710-4261-a9b6-20c5ffa06822 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:10.837444 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad81c2-9710-4261-a9b6-20c5ffa06822 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:10.837717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad81c2-9710-4261-a9b6-20c5ffa06822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:10.870252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:10.914505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2fad81c2-9710-4261-a9b6-20c5ffa06822 None None] MFA Rules not processed for user `ea77fa8f5b754a2287f0639717f6e57a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:11:10.915747 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2fad81c2-9710-4261-a9b6-20c5ffa06822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:10.990725 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 747/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Fri Sep 9 13:11:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:11:10.999742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f17a8-1996-4cd2-afad-c54b8fe50616 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:11:11.001268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-807f17a8-1996-4cd2-afad-c54b8fe50616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:11.051919 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:11.107542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f17a8-1996-4cd2-afad-c54b8fe50616 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:11:11.180543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209abdce-ce05-4fd7-b209-c01d808c717f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:11:11.181156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209abdce-ce05-4fd7-b209-c01d808c717f service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:11.181436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209abdce-ce05-4fd7-b209-c01d808c717f service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:11.181706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209abdce-ce05-4fd7-b209-c01d808c717f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:11.183325 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-209abdce-ce05-4fd7-b209-c01d808c717f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:11.282436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209abdce-ce05-4fd7-b209-c01d808c717f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:11:11.283817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209abdce-ce05-4fd7-b209-c01d808c717f service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:11:11.287335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:11.302373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 758/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:11:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:11:11.348911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1f14a-8cc2-4c29-a21e-5bf80c4cc0f2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:11.349199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1f14a-8cc2-4c29-a21e-5bf80c4cc0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:11.349478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1f14a-8cc2-4c29-a21e-5bf80c4cc0f2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:11.350754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 748/1506] 10.222.0.17 () {52 vars in 830 bytes} [Fri Sep 9 13:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:11:11.354449 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209abdce-ce05-4fd7-b209-c01d808c717f service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:11:11.355684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-209abdce-ce05-4fd7-b209-c01d808c717f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:11.455740 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209abdce-ce05-4fd7-b209-c01d808c717f service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:11:11.548988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:11:11.549598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:11.549798 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:11.550065 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] PATH_INFO: `/v3/users/213d942b9f164d9eab0c2631f1666b0f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:11.563412 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=213d942b9f164d9eab0c2631f1666b0f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:11:11.564821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:11:11.564821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Sep 09 13:11:11.564821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: Sep 09 13:11:11.566026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:11:11.576174 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:11.636370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '213d942b9f164d9eab0c2631f1666b0f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:11.645207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '213d942b9f164d9eab0c2631f1666b0f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:11.645638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '213d942b9f164d9eab0c2631f1666b0f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:11.669967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '213d942b9f164d9eab0c2631f1666b0f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:11.683869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '213d942b9f164d9eab0c2631f1666b0f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:11.690059 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 759/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:11:11 2022] DELETE /identity/v3/users/213d942b9f164d9eab0c2631f1666b0f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:11:11.694541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae0913-139c-45f7-aba6-afdf82f48dd3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:11.694828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae0913-139c-45f7-aba6-afdf82f48dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:11.695133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae0913-139c-45f7-aba6-afdf82f48dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:11.753160 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-07ae0913-139c-45f7-aba6-afdf82f48dd3 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:11:11.754408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-07ae0913-139c-45f7-aba6-afdf82f48dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:11.828794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:11.841240 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 749/1508] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:11:11.851044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:11:11.852292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:11.953391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8a361-b43e-401f-8206-8bbeda9d9698 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:11:12.009244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:12.031001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721f9a1-37d1-43e3-b90f-bf572063f67c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:11:12.031644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721f9a1-37d1-43e3-b90f-bf572063f67c service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:12.031924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721f9a1-37d1-43e3-b90f-bf572063f67c service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:12.032198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721f9a1-37d1-43e3-b90f-bf572063f67c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:12.033794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3721f9a1-37d1-43e3-b90f-bf572063f67c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:12.141003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721f9a1-37d1-43e3-b90f-bf572063f67c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:11:12.142363 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721f9a1-37d1-43e3-b90f-bf572063f67c service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:11:12.162099 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 760/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:11:12.505011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae0913-139c-45f7-aba6-afdf82f48dd3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:11:12.506250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-07ae0913-139c-45f7-aba6-afdf82f48dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:12.518149 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae0913-139c-45f7-aba6-afdf82f48dd3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:11:12.617128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:11:12.617766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:12.618045 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:12.618316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b admin admin] PATH_INFO: `/v3/projects/fe5a65456d5b487bb7d478e66cb30833` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:12.628923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5a65456d5b487bb7d478e66cb30833)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:11:12.629949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:11:12.629949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 09 13:11:12.629949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: Sep 09 13:11:12.631171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:11:12.690837 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5a65456d5b487bb7d478e66cb30833', 'request_id': 'req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:12.696870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5a65456d5b487bb7d478e66cb30833', 'request_id': 'req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:12.709750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5a65456d5b487bb7d478e66cb30833', 'request_id': 'req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:12.737778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b admin admin] The token cache is being invalidate because project fe5a65456d5b487bb7d478e66cb30833 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:11:12.738070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:12.738500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dba61969-b9c9-4a7d-91ea-b7fa62ca351b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:12.739950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 750/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:11:12 2022] DELETE /identity/v3/projects/fe5a65456d5b487bb7d478e66cb30833 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:11:22.158713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=62 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:22.735715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:12:40.609602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcc2cb-7df9-4e4f-af52-23381709f49d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:40.610173 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcc2cb-7df9-4e4f-af52-23381709f49d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:40.610173 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcc2cb-7df9-4e4f-af52-23381709f49d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:40.611460 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 13:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:12:40.616442 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d889e95-96ef-41b0-9d9e-9662e67a4d13 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:40.616902 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d889e95-96ef-41b0-9d9e-9662e67a4d13 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:40.617009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d889e95-96ef-41b0-9d9e-9662e67a4d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:40.697912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8d889e95-96ef-41b0-9d9e-9662e67a4d13 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:12:40.699167 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8d889e95-96ef-41b0-9d9e-9662e67a4d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:12:40.808500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 13:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:12:40.962166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcc2cb-7df9-4e4f-af52-23381709f49d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:12:40.964390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-98bcc2cb-7df9-4e4f-af52-23381709f49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:12:41.083178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcc2cb-7df9-4e4f-af52-23381709f49d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:12:41.088685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fce53-b002-4f71-9242-196e183b104c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:12:41.089294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fce53-b002-4f71-9242-196e183b104c service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:41.089576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fce53-b002-4f71-9242-196e183b104c service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:41.089852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fce53-b002-4f71-9242-196e183b104c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:41.097798 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fce53-b002-4f71-9242-196e183b104c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:12:41.099470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fce53-b002-4f71-9242-196e183b104c service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:12:41.108991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 762/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:12:42.211851 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307ac21-cd91-45a3-b684-52508082b3eb None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:42.212399 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307ac21-cd91-45a3-b684-52508082b3eb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:42.212399 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307ac21-cd91-45a3-b684-52508082b3eb None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:42.213760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 13:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:12:42.220768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b5e48-94b9-49a8-a6da-7067ed6bbeae None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:42.221227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b5e48-94b9-49a8-a6da-7067ed6bbeae None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:42.221323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b5e48-94b9-49a8-a6da-7067ed6bbeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:42.241904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1c8b5e48-94b9-49a8-a6da-7067ed6bbeae None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:12:42.243144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1c8b5e48-94b9-49a8-a6da-7067ed6bbeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:12:42.251142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 13:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:12:42.404315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307ac21-cd91-45a3-b684-52508082b3eb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:12:42.418416 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307ac21-cd91-45a3-b684-52508082b3eb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:12:42.423867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc056ab-bf6b-4e61-89a7-bea670df6a29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:12:42.424461 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc056ab-bf6b-4e61-89a7-bea670df6a29 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:42.424740 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc056ab-bf6b-4e61-89a7-bea670df6a29 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:42.425012 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc056ab-bf6b-4e61-89a7-bea670df6a29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:42.433759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc056ab-bf6b-4e61-89a7-bea670df6a29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:12:42.435374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc056ab-bf6b-4e61-89a7-bea670df6a29 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:12:42.445201 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 753/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:12:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:12:42.723766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df4a69-2637-435b-a57a-809212158012 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:42.724056 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df4a69-2637-435b-a57a-809212158012 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:42.724335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df4a69-2637-435b-a57a-809212158012 None None] PATH_INFO: `/v3` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:42.725634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 764/1517] 10.222.0.49 () {54 vars in 925 bytes} [Fri Sep 9 13:12:42 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:12:42.731811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533d294-b722-4350-bbaf-b4459f4d127e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:42.732101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533d294-b722-4350-bbaf-b4459f4d127e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:42.732381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533d294-b722-4350-bbaf-b4459f4d127e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:42.792525 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5533d294-b722-4350-bbaf-b4459f4d127e None None] MFA Rules not processed for user `dbc7cef7b714463697dc1ee507e60be1`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:12:42.793749 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5533d294-b722-4350-bbaf-b4459f4d127e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:12:42.810918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:12:42.865968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 754/1518] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Sep 9 13:12:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:12:46.362181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f8fe3-a689-4eb2-a1f6-b58240097d20 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:46.362625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f8fe3-a689-4eb2-a1f6-b58240097d20 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:46.362753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f8fe3-a689-4eb2-a1f6-b58240097d20 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:46.364187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 765/1519] 10.222.0.17 () {54 vars in 961 bytes} [Fri Sep 9 13:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:12:46.369393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac889253-b06c-40e5-a790-69121ab8fc46 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:46.369880 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac889253-b06c-40e5-a790-69121ab8fc46 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:46.369993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac889253-b06c-40e5-a790-69121ab8fc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:46.389910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ac889253-b06c-40e5-a790-69121ab8fc46 None None] MFA Rules not processed for user `dbc7cef7b714463697dc1ee507e60be1`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:12:46.391153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ac889253-b06c-40e5-a790-69121ab8fc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:12:46.403635 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 755/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Sep 9 13:12:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:12:46.412028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f8fe3-a689-4eb2-a1f6-b58240097d20 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:12:46.429523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f8fe3-a689-4eb2-a1f6-b58240097d20 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:12:46.434896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9d77c-1585-49d9-b65a-da7412c242a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:12:46.435531 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d77c-1585-49d9-b65a-da7412c242a5 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:46.435806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d77c-1585-49d9-b65a-da7412c242a5 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:46.436073 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d77c-1585-49d9-b65a-da7412c242a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:46.445207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9d77c-1585-49d9-b65a-da7412c242a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:12:46.446775 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9d77c-1585-49d9-b65a-da7412c242a5 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:12:46.456194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 766/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:12:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:12:53.916382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09936c4f-2caa-4e54-8e26-639c536c154d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:53.916915 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09936c4f-2caa-4e54-8e26-639c536c154d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:53.916915 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09936c4f-2caa-4e54-8e26-639c536c154d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:53.918216 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 13:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:12:53.924573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487959c1-2724-4b0b-9ce2-b6589af049f4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:53.924989 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487959c1-2724-4b0b-9ce2-b6589af049f4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:53.925143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487959c1-2724-4b0b-9ce2-b6589af049f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:53.945472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-487959c1-2724-4b0b-9ce2-b6589af049f4 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:12:53.946670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-487959c1-2724-4b0b-9ce2-b6589af049f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:12:53.954405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 13:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:12:54.109155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09936c4f-2caa-4e54-8e26-639c536c154d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:12:54.124310 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09936c4f-2caa-4e54-8e26-639c536c154d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:12:54.129698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259945c-acde-4e8f-b385-cf17b860ddab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:12:54.130567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259945c-acde-4e8f-b385-cf17b860ddab service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:54.130848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259945c-acde-4e8f-b385-cf17b860ddab service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:54.131158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259945c-acde-4e8f-b385-cf17b860ddab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:54.139057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259945c-acde-4e8f-b385-cf17b860ddab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:12:54.140693 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259945c-acde-4e8f-b385-cf17b860ddab service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:12:54.149986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 757/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:12:55.229150 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b85a4-32c7-4e0a-987c-d4d13bfc05db None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:55.229988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b85a4-32c7-4e0a-987c-d4d13bfc05db None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:55.229988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b85a4-32c7-4e0a-987c-d4d13bfc05db None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:55.231381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 13:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:12:55.237553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bff438-65d8-4669-b086-32d797a99027 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:55.238000 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bff438-65d8-4669-b086-32d797a99027 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:55.238101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bff438-65d8-4669-b086-32d797a99027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:55.258761 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-05bff438-65d8-4669-b086-32d797a99027 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:12:55.259975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-05bff438-65d8-4669-b086-32d797a99027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:12:55.267895 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 13:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:12:55.420830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b85a4-32c7-4e0a-987c-d4d13bfc05db None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:12:55.434949 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b85a4-32c7-4e0a-987c-d4d13bfc05db None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:12:55.440411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80903553-255b-4875-8de6-0c4ce4afe10a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:12:55.441013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80903553-255b-4875-8de6-0c4ce4afe10a service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:55.441290 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80903553-255b-4875-8de6-0c4ce4afe10a service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:55.441563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80903553-255b-4875-8de6-0c4ce4afe10a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:55.450079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80903553-255b-4875-8de6-0c4ce4afe10a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:12:55.451696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80903553-255b-4875-8de6-0c4ce4afe10a service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:12:55.461588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 769/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:12:59.612277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad51a5-0d99-48b7-8a72-d6ae97a531ec None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:59.612780 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad51a5-0d99-48b7-8a72-d6ae97a531ec None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:59.612882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad51a5-0d99-48b7-8a72-d6ae97a531ec None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:59.614183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:12:59.620233 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab447ec-1a64-4b14-b8c5-8beeca632c26 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:12:59.620679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab447ec-1a64-4b14-b8c5-8beeca632c26 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:12:59.620784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab447ec-1a64-4b14-b8c5-8beeca632c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:12:59.640721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1ab447ec-1a64-4b14-b8c5-8beeca632c26 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:12:59.641926 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1ab447ec-1a64-4b14-b8c5-8beeca632c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:12:59.650004 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:12:59.937610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad51a5-0d99-48b7-8a72-d6ae97a531ec None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:12:59.939367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aaad51a5-0d99-48b7-8a72-d6ae97a531ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:13:00.017359 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad51a5-0d99-48b7-8a72-d6ae97a531ec None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:13:00.020950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eae850-c49f-476f-83a6-146661f0f00b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:13:00.021545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae850-c49f-476f-83a6-146661f0f00b service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:13:00.021823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae850-c49f-476f-83a6-146661f0f00b service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:13:00.022095 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae850-c49f-476f-83a6-146661f0f00b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:13:00.030909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eae850-c49f-476f-83a6-146661f0f00b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:13:00.032365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eae850-c49f-476f-83a6-146661f0f00b service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:13:00.042264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 760/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:12:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:13:02.182236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc86d22-963f-4ca9-88ea-03fb3dfcb43f None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:13:02.182826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc86d22-963f-4ca9-88ea-03fb3dfcb43f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:13:02.182826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc86d22-963f-4ca9-88ea-03fb3dfcb43f None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:13:02.184169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:13:02.189468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd9718-c0c0-406c-b0c1-671a9017613d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:13:02.189863 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd9718-c0c0-406c-b0c1-671a9017613d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:13:02.190028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd9718-c0c0-406c-b0c1-671a9017613d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:13:02.210613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-52dd9718-c0c0-406c-b0c1-671a9017613d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:13:02.211868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-52dd9718-c0c0-406c-b0c1-671a9017613d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:13:02.219674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:13:02.514731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc86d22-963f-4ca9-88ea-03fb3dfcb43f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:13:02.520250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=75 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:13:02.527474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc86d22-963f-4ca9-88ea-03fb3dfcb43f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:13:02.531168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea8a1b-d2e0-482a-9598-bab6372ca9fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:13:02.531768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea8a1b-d2e0-482a-9598-bab6372ca9fc service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:13:02.532046 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea8a1b-d2e0-482a-9598-bab6372ca9fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:13:02.532318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea8a1b-d2e0-482a-9598-bab6372ca9fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:13:02.541474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea8a1b-d2e0-482a-9598-bab6372ca9fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:13:02.542887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea8a1b-d2e0-482a-9598-bab6372ca9fc service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:13:02.552055 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 772/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:13:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:13:06.270875 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f193a2-e43f-4ff1-833f-734dfefd8f50 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:13:06.272009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f193a2-e43f-4ff1-833f-734dfefd8f50 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:13:06.272009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f193a2-e43f-4ff1-833f-734dfefd8f50 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:13:06.273576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:13:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:13:06.278824 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe74899-958f-47f6-953c-79799cce396f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:13:06.279291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe74899-958f-47f6-953c-79799cce396f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:13:06.279390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe74899-958f-47f6-953c-79799cce396f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:13:06.299481 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5fe74899-958f-47f6-953c-79799cce396f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:13:06.300686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5fe74899-958f-47f6-953c-79799cce396f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:13:06.308576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:13:06.613351 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f193a2-e43f-4ff1-833f-734dfefd8f50 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:13:06.615606 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f5f193a2-e43f-4ff1-833f-734dfefd8f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:13:06.680511 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:13:06.694825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f193a2-e43f-4ff1-833f-734dfefd8f50 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:13:06.698367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35512a-7ee2-45bd-b192-44a8ffbc97b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:13:06.698964 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35512a-7ee2-45bd-b192-44a8ffbc97b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:13:06.699270 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35512a-7ee2-45bd-b192-44a8ffbc97b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:13:06.699539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35512a-7ee2-45bd-b192-44a8ffbc97b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:13:06.708284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35512a-7ee2-45bd-b192-44a8ffbc97b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:13:06.709987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35512a-7ee2-45bd-b192-44a8ffbc97b4 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:13:06.719182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 763/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:13:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:13:08.860919 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695ad59-23a7-4a05-a189-3f96eeeda8be None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:13:08.861485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695ad59-23a7-4a05-a189-3f96eeeda8be None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:13:08.861485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695ad59-23a7-4a05-a189-3f96eeeda8be None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:13:08.862804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 13:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:13:08.868663 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f37c2e-fe4c-4d0b-9167-04455d07d858 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:13:08.869093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f37c2e-fe4c-4d0b-9167-04455d07d858 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:13:08.869206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f37c2e-fe4c-4d0b-9167-04455d07d858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:13:08.889569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a7f37c2e-fe4c-4d0b-9167-04455d07d858 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:13:08.890802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a7f37c2e-fe4c-4d0b-9167-04455d07d858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:13:08.903642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 13:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:13:09.202841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695ad59-23a7-4a05-a189-3f96eeeda8be None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:13:09.204652 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a695ad59-23a7-4a05-a189-3f96eeeda8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:13:09.217605 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695ad59-23a7-4a05-a189-3f96eeeda8be None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:13:09.221187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017833c-b6cb-454c-af2b-ab33a7fe602e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:13:09.221788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017833c-b6cb-454c-af2b-ab33a7fe602e service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:13:09.222066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017833c-b6cb-454c-af2b-ab33a7fe602e service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:13:09.222338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017833c-b6cb-454c-af2b-ab33a7fe602e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:13:09.231936 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017833c-b6cb-454c-af2b-ab33a7fe602e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:13:09.233360 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017833c-b6cb-454c-af2b-ab33a7fe602e service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:13:09.243337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 775/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:13:18.884430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=11 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:13:19.238474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=14 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:32.714885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbc935-a845-4350-b5e5-f4b05161473c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:32.716825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbc935-a845-4350-b5e5-f4b05161473c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:32.717324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbc935-a845-4350-b5e5-f4b05161473c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:32.748323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c1dbc935-a845-4350-b5e5-f4b05161473c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:32.750025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c1dbc935-a845-4350-b5e5-f4b05161473c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:32.759075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 765/1540] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:32.769061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47276324-6b6f-4c22-b10f-9292812f0f5f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:32.769659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47276324-6b6f-4c22-b10f-9292812f0f5f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:32.769659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47276324-6b6f-4c22-b10f-9292812f0f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:32.799909 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-47276324-6b6f-4c22-b10f-9292812f0f5f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:32.801677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-47276324-6b6f-4c22-b10f-9292812f0f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:32.813269 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 776/1541] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:32.817791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbc935-a845-4350-b5e5-f4b05161473c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:32.829292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbc935-a845-4350-b5e5-f4b05161473c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:32.837169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f608a58-0fe2-4c59-a57c-9720d56810ba None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:32.838163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f608a58-0fe2-4c59-a57c-9720d56810ba None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:32.838683 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f608a58-0fe2-4c59-a57c-9720d56810ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:32.838917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a0b99-6507-4cb6-8bb1-99fd14931e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:32.839590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a0b99-6507-4cb6-8bb1-99fd14931e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:32.839873 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a0b99-6507-4cb6-8bb1-99fd14931e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:32.840707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a0b99-6507-4cb6-8bb1-99fd14931e43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:32.841587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a0b99-6507-4cb6-8bb1-99fd14931e43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:14:32.841895 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a0b99-6507-4cb6-8bb1-99fd14931e43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:32.843582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a0b99-6507-4cb6-8bb1-99fd14931e43 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:32.865601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 766/1542] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:14:32.875096 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a0b99-6507-4cb6-8bb1-99fd14931e43 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:32.876108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8f608a58-0fe2-4c59-a57c-9720d56810ba None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:32.877353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8f608a58-0fe2-4c59-a57c-9720d56810ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:32.889699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a0b99-6507-4cb6-8bb1-99fd14931e43 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:32.892741 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 777/1543] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:14:32.900156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e145cd-7a1a-4026-b263-897bf2545628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:32.900791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e145cd-7a1a-4026-b263-897bf2545628 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:32.901070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e145cd-7a1a-4026-b263-897bf2545628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:32.901253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3ac72-be00-475a-b70d-dd97655fd2ac None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:32.901368 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e145cd-7a1a-4026-b263-897bf2545628 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:32.901529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3ac72-be00-475a-b70d-dd97655fd2ac None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:32.901801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3ac72-be00-475a-b70d-dd97655fd2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:32.901977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e145cd-7a1a-4026-b263-897bf2545628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:32.903569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-23e145cd-7a1a-4026-b263-897bf2545628 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:14:32.903569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 09 13:14:32.903569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: Sep 09 13:14:32.905020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e145cd-7a1a-4026-b263-897bf2545628 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:32.932368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6ff3ac72-be00-475a-b70d-dd97655fd2ac None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:32.933617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6ff3ac72-be00-475a-b70d-dd97655fd2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:32.952923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 778/1544] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:14:32.958122 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3ac72-be00-475a-b70d-dd97655fd2ac None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:32.973322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3ac72-be00-475a-b70d-dd97655fd2ac None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:32.989330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039c1ed-9e34-4810-a8a4-3617c7a39467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:32.989975 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039c1ed-9e34-4810-a8a4-3617c7a39467 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:32.990218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 767/1545] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:14:32 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:32.990348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039c1ed-9e34-4810-a8a4-3617c7a39467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:32.990552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039c1ed-9e34-4810-a8a4-3617c7a39467 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:32.991201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039c1ed-9e34-4810-a8a4-3617c7a39467 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:14:32.991606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039c1ed-9e34-4810-a8a4-3617c7a39467 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:32.991731 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37a0f3-4993-4c6b-90a8-760fb3dc5765 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:32.991818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37a0f3-4993-4c6b-90a8-760fb3dc5765 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:32.992144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37a0f3-4993-4c6b-90a8-760fb3dc5765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:32.992523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-b039c1ed-9e34-4810-a8a4-3617c7a39467 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:14:32.992523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Sep 09 13:14:32.992523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: Sep 09 13:14:32.993418 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039c1ed-9e34-4810-a8a4-3617c7a39467 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.031078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 779/1546] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:14:33.031078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039c1ed-9e34-4810-a8a4-3617c7a39467 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.032636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fa37a0f3-4993-4c6b-90a8-760fb3dc5765 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:33.034264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fa37a0f3-4993-4c6b-90a8-760fb3dc5765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:33.053982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039c1ed-9e34-4810-a8a4-3617c7a39467 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.142234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 768/1547] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:14:33.143582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6742059-2555-4d35-bc5e-752e708385d0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.143866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6742059-2555-4d35-bc5e-752e708385d0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.144139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6742059-2555-4d35-bc5e-752e708385d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.164064 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f951a6b-75ef-4b90-a2b6-59fc24e3c4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.164691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f951a6b-75ef-4b90-a2b6-59fc24e3c4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.164970 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f951a6b-75ef-4b90-a2b6-59fc24e3c4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.165245 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f951a6b-75ef-4b90-a2b6-59fc24e3c4a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.165885 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f951a6b-75ef-4b90-a2b6-59fc24e3c4a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.166904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-0f951a6b-75ef-4b90-a2b6-59fc24e3c4a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:14:33.166904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Sep 09 13:14:33.166904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: Sep 09 13:14:33.168153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f951a6b-75ef-4b90-a2b6-59fc24e3c4a2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.188543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a6742059-2555-4d35-bc5e-752e708385d0 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:33.189775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a6742059-2555-4d35-bc5e-752e708385d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:33.202139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 769/1548] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:33.203149 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6742059-2555-4d35-bc5e-752e708385d0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.209016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 780/1549] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:33.210335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bd32d-2385-4d9a-a43f-bc359b3625c7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.210623 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bd32d-2385-4d9a-a43f-bc359b3625c7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.210894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bd32d-2385-4d9a-a43f-bc359b3625c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.216103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6742059-2555-4d35-bc5e-752e708385d0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.221907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13dc64-6a8a-4587-b451-9f834416f198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.222963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13dc64-6a8a-4587-b451-9f834416f198 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.223272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13dc64-6a8a-4587-b451-9f834416f198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.223601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13dc64-6a8a-4587-b451-9f834416f198 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.224185 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13dc64-6a8a-4587-b451-9f834416f198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.226104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13dc64-6a8a-4587-b451-9f834416f198 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.233345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-220bd32d-2385-4d9a-a43f-bc359b3625c7 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:33.234535 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-220bd32d-2385-4d9a-a43f-bc359b3625c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:33.242638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 781/1550] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:33.243899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d078eb6-4eba-4529-87b0-6349c112c69f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.244178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d078eb6-4eba-4529-87b0-6349c112c69f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.244446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d078eb6-4eba-4529-87b0-6349c112c69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.255229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 770/1551] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/projects => generated 357 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:33.256481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc147e23-a917-4760-9c0d-9bb597191593 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.256764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc147e23-a917-4760-9c0d-9bb597191593 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.257037 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc147e23-a917-4760-9c0d-9bb597191593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.264796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2d078eb6-4eba-4529-87b0-6349c112c69f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:33.265986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2d078eb6-4eba-4529-87b0-6349c112c69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:33.277005 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bc147e23-a917-4760-9c0d-9bb597191593 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:33.278963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bc147e23-a917-4760-9c0d-9bb597191593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:33.324248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 782/1552] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:33.326162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a5b36-17a0-4c41-b635-c1d83c55e511 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.326445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a5b36-17a0-4c41-b635-c1d83c55e511 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.326714 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a5b36-17a0-4c41-b635-c1d83c55e511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.342589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 771/1553] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:33.343885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee015b-71da-475c-9fcf-d32bcc970931 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.344196 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee015b-71da-475c-9fcf-d32bcc970931 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.344471 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee015b-71da-475c-9fcf-d32bcc970931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.347324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-111a5b36-17a0-4c41-b635-c1d83c55e511 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:33.348663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-111a5b36-17a0-4c41-b635-c1d83c55e511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:33.357057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 783/1554] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:33.358103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a5b36-17a0-4c41-b635-c1d83c55e511 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.364531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-14ee015b-71da-475c-9fcf-d32bcc970931 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:33.365875 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-14ee015b-71da-475c-9fcf-d32bcc970931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:33.367857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a5b36-17a0-4c41-b635-c1d83c55e511 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.372512 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee083a-aee3-420e-9daa-d915b42e80b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.373111 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee083a-aee3-420e-9daa-d915b42e80b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.373388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee083a-aee3-420e-9daa-d915b42e80b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.373658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee083a-aee3-420e-9daa-d915b42e80b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.374297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee083a-aee3-420e-9daa-d915b42e80b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.374431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 772/1555] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:33.375626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c96a3e-e560-40e7-9d90-0ccb16d81a7c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.375907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c96a3e-e560-40e7-9d90-0ccb16d81a7c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.376034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee083a-aee3-420e-9daa-d915b42e80b6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.376183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c96a3e-e560-40e7-9d90-0ccb16d81a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.387877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 784/1556] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:33.389250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d2afb-eddb-4ddd-bb04-ebd650973917 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.389533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d2afb-eddb-4ddd-bb04-ebd650973917 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.389808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d2afb-eddb-4ddd-bb04-ebd650973917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.396935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e6c96a3e-e560-40e7-9d90-0ccb16d81a7c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:33.398150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e6c96a3e-e560-40e7-9d90-0ccb16d81a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:33.406554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 773/1557] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:33.407413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c96a3e-e560-40e7-9d90-0ccb16d81a7c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.409850 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-eb8d2afb-eddb-4ddd-bb04-ebd650973917 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:33.411449 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-eb8d2afb-eddb-4ddd-bb04-ebd650973917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:33.419237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c96a3e-e560-40e7-9d90-0ccb16d81a7c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.421077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 785/1558] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:33.421963 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d2afb-eddb-4ddd-bb04-ebd650973917 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.426151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6816e-674f-43b3-99c7-60f801a2ba19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.426788 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6816e-674f-43b3-99c7-60f801a2ba19 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.427092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6816e-674f-43b3-99c7-60f801a2ba19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.427370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6816e-674f-43b3-99c7-60f801a2ba19 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.427998 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6816e-674f-43b3-99c7-60f801a2ba19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.429963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6816e-674f-43b3-99c7-60f801a2ba19 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.431278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d2afb-eddb-4ddd-bb04-ebd650973917 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.436557 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6e587-229b-4c9d-9e3d-a4fe12a87b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.437147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6e587-229b-4c9d-9e3d-a4fe12a87b39 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.437427 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6e587-229b-4c9d-9e3d-a4fe12a87b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.437704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6e587-229b-4c9d-9e3d-a4fe12a87b39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.438346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6e587-229b-4c9d-9e3d-a4fe12a87b39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:14:33.438653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6e587-229b-4c9d-9e3d-a4fe12a87b39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.440167 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:33.440302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6e587-229b-4c9d-9e3d-a4fe12a87b39 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.453918 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 786/1559] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:14:33.454579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 774/1560] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/users => generated 279 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:33.455718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05921772-7fc6-4883-993f-4d8786facdf1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.456030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1f5b5-7945-4612-90c3-80d9d541a784 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.456159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05921772-7fc6-4883-993f-4d8786facdf1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.456327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05921772-7fc6-4883-993f-4d8786facdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.456470 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1f5b5-7945-4612-90c3-80d9d541a784 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.456801 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1f5b5-7945-4612-90c3-80d9d541a784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.481533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-05921772-7fc6-4883-993f-4d8786facdf1 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:33.482706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-05921772-7fc6-4883-993f-4d8786facdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:33.487245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e3a1f5b5-7945-4612-90c3-80d9d541a784 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:33.488437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e3a1f5b5-7945-4612-90c3-80d9d541a784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:33.490830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 787/1561] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:33.491629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05921772-7fc6-4883-993f-4d8786facdf1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.497098 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 775/1562] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:33.501524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1f5b5-7945-4612-90c3-80d9d541a784 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.501667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05921772-7fc6-4883-993f-4d8786facdf1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.506169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401b3ac-d365-410b-9589-f3e671e51f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.506761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401b3ac-d365-410b-9589-f3e671e51f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.507034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401b3ac-d365-410b-9589-f3e671e51f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.507331 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401b3ac-d365-410b-9589-f3e671e51f0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.508078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401b3ac-d365-410b-9589-f3e671e51f0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:14:33.508375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401b3ac-d365-410b-9589-f3e671e51f0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.509988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401b3ac-d365-410b-9589-f3e671e51f0d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.511631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:33.511760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1f5b5-7945-4612-90c3-80d9d541a784 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.516986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a5696-f3c2-45cb-b4da-1b27eb2efe70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.517999 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a5696-f3c2-45cb-b4da-1b27eb2efe70 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.517999 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a5696-f3c2-45cb-b4da-1b27eb2efe70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.518225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a5696-f3c2-45cb-b4da-1b27eb2efe70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.518829 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a5696-f3c2-45cb-b4da-1b27eb2efe70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.520598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a5696-f3c2-45cb-b4da-1b27eb2efe70 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.530516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 788/1563] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:14:33.530516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0b0f4-9003-4d9d-af72-926e8c46a8e2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.530516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0b0f4-9003-4d9d-af72-926e8c46a8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.530516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0b0f4-9003-4d9d-af72-926e8c46a8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.531383 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 776/1564] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:33.532425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a5696-f3c2-45cb-b4da-1b27eb2efe70 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.546287 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a5696-f3c2-45cb-b4da-1b27eb2efe70 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.548854 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0ba0b0f4-9003-4d9d-af72-926e8c46a8e2 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:33.550097 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0ba0b0f4-9003-4d9d-af72-926e8c46a8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:33.550977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ad316-eaea-4b81-a018-541c6e7ac9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.551608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ad316-eaea-4b81-a018-541c6e7ac9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.551884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ad316-eaea-4b81-a018-541c6e7ac9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.552151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ad316-eaea-4b81-a018-541c6e7ac9e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.552823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ad316-eaea-4b81-a018-541c6e7ac9e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:14:33.553124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ad316-eaea-4b81-a018-541c6e7ac9e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.554700 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ad316-eaea-4b81-a018-541c6e7ac9e2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.558221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 789/1565] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:33.562878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0b0f4-9003-4d9d-af72-926e8c46a8e2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.571202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 777/1566] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:14:33.571348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0b0f4-9003-4d9d-af72-926e8c46a8e2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.571464 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ad316-eaea-4b81-a018-541c6e7ac9e2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.578772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ad316-eaea-4b81-a018-541c6e7ac9e2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.580343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7ae61-5255-4002-bc8f-53963582c629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.580958 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7ae61-5255-4002-bc8f-53963582c629 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.581235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7ae61-5255-4002-bc8f-53963582c629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.581506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7ae61-5255-4002-bc8f-53963582c629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.582184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7ae61-5255-4002-bc8f-53963582c629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.583331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba89d9-6ee4-47bf-b4ce-999ab3e42138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.583885 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7ae61-5255-4002-bc8f-53963582c629 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.583994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba89d9-6ee4-47bf-b4ce-999ab3e42138 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.584166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba89d9-6ee4-47bf-b4ce-999ab3e42138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.584433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba89d9-6ee4-47bf-b4ce-999ab3e42138 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.585049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba89d9-6ee4-47bf-b4ce-999ab3e42138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.586994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba89d9-6ee4-47bf-b4ce-999ab3e42138 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.595226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 790/1567] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:33.599411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7ae61-5255-4002-bc8f-53963582c629 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.611369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7ae61-5255-4002-bc8f-53963582c629 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.616625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050026d-b12b-4268-844b-a72b9f7ccb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.617251 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050026d-b12b-4268-844b-a72b9f7ccb05 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.617523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050026d-b12b-4268-844b-a72b9f7ccb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.617805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050026d-b12b-4268-844b-a72b9f7ccb05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.618446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050026d-b12b-4268-844b-a72b9f7ccb05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:14:33.618747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050026d-b12b-4268-844b-a72b9f7ccb05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.619331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 778/1568] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:33.620322 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba89d9-6ee4-47bf-b4ce-999ab3e42138 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.623615 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050026d-b12b-4268-844b-a72b9f7ccb05 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.636782 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 791/1569] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:14:33.637424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050026d-b12b-4268-844b-a72b9f7ccb05 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.647257 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050026d-b12b-4268-844b-a72b9f7ccb05 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.686735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba89d9-6ee4-47bf-b4ce-999ab3e42138 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.691502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ff68b-bca9-4976-9201-c3b55b963cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.692117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ff68b-bca9-4976-9201-c3b55b963cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.692392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ff68b-bca9-4976-9201-c3b55b963cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.692666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ff68b-bca9-4976-9201-c3b55b963cc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.693320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ff68b-bca9-4976-9201-c3b55b963cc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:14:33.693657 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ff68b-bca9-4976-9201-c3b55b963cc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.695295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ff68b-bca9-4976-9201-c3b55b963cc0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.704676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4378e5a-f29a-4e47-81d4-7014cd6c92b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.705407 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4378e5a-f29a-4e47-81d4-7014cd6c92b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.705684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4378e5a-f29a-4e47-81d4-7014cd6c92b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.705980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4378e5a-f29a-4e47-81d4-7014cd6c92b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.706603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4378e5a-f29a-4e47-81d4-7014cd6c92b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.706763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 779/1570] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:14:33.707597 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ff68b-bca9-4976-9201-c3b55b963cc0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.709127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4378e5a-f29a-4e47-81d4-7014cd6c92b3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.719504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ff68b-bca9-4976-9201-c3b55b963cc0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.724171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ae9de-2fd8-48b2-8dbb-d9009f712cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.724824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ae9de-2fd8-48b2-8dbb-d9009f712cf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.725102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ae9de-2fd8-48b2-8dbb-d9009f712cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.725377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ae9de-2fd8-48b2-8dbb-d9009f712cf2 admin admin] PATH_INFO: `/v3/projects/80ecff6bf26f4cbeadb78b7272febb35/users/efeb17c5a72b4e9294a63cf185737f3f/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.741748 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 792/1571] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:33.741748 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4378e5a-f29a-4e47-81d4-7014cd6c92b3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.745316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ae9de-2fd8-48b2-8dbb-d9009f712cf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ecff6bf26f4cbeadb78b7272febb35, user_id=efeb17c5a72b4e9294a63cf185737f3f, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.746465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-7c5ae9de-2fd8-48b2-8dbb-d9009f712cf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:14:33.746465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 09 13:14:33.746465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: Sep 09 13:14:33.748641 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ae9de-2fd8-48b2-8dbb-d9009f712cf2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.766573 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 780/1572] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:33 2022] PUT /identity/v3/projects/80ecff6bf26f4cbeadb78b7272febb35/users/efeb17c5a72b4e9294a63cf185737f3f/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:33.767506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ae9de-2fd8-48b2-8dbb-d9009f712cf2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.802599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4378e5a-f29a-4e47-81d4-7014cd6c92b3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.831586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ae9de-2fd8-48b2-8dbb-d9009f712cf2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.836534 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4c57b-759e-47f1-927a-3ef757d7b571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.837174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4c57b-759e-47f1-927a-3ef757d7b571 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.837458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4c57b-759e-47f1-927a-3ef757d7b571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.837735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4c57b-759e-47f1-927a-3ef757d7b571 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.838739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4c57b-759e-47f1-927a-3ef757d7b571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.841302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4c57b-759e-47f1-927a-3ef757d7b571 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.851700 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:33.855981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:33.859387 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8769ee-ff53-4b01-9b2c-736b1817de18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.860271 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8769ee-ff53-4b01-9b2c-736b1817de18 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.860549 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8769ee-ff53-4b01-9b2c-736b1817de18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.860816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8769ee-ff53-4b01-9b2c-736b1817de18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.861472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8769ee-ff53-4b01-9b2c-736b1817de18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:14:33.861783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8769ee-ff53-4b01-9b2c-736b1817de18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.863810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8769ee-ff53-4b01-9b2c-736b1817de18 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.869886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 781/1573] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:33.870886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4c57b-759e-47f1-927a-3ef757d7b571 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.878171 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 793/1574] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:14:33.879182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8769ee-ff53-4b01-9b2c-736b1817de18 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.889563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8769ee-ff53-4b01-9b2c-736b1817de18 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.936083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4c57b-759e-47f1-927a-3ef757d7b571 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.940791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c277a2d-32e1-4f7c-a90b-42ffae301e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.941390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c277a2d-32e1-4f7c-a90b-42ffae301e4e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.941673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c277a2d-32e1-4f7c-a90b-42ffae301e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.941947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c277a2d-32e1-4f7c-a90b-42ffae301e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.942617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c277a2d-32e1-4f7c-a90b-42ffae301e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.944258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c277a2d-32e1-4f7c-a90b-42ffae301e4e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.947182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579cbb3f-1636-4dc7-9bec-44499ad33ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.947804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579cbb3f-1636-4dc7-9bec-44499ad33ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.948086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579cbb3f-1636-4dc7-9bec-44499ad33ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.948364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579cbb3f-1636-4dc7-9bec-44499ad33ad6 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.949026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579cbb3f-1636-4dc7-9bec-44499ad33ad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.951269 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579cbb3f-1636-4dc7-9bec-44499ad33ad6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.954063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 782/1575] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:33.954858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c277a2d-32e1-4f7c-a90b-42ffae301e4e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.964886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c277a2d-32e1-4f7c-a90b-42ffae301e4e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.970731 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee7fca-c75a-4b5f-9c73-033a84cebeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.971364 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee7fca-c75a-4b5f-9c73-033a84cebeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.971647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee7fca-c75a-4b5f-9c73-033a84cebeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.971924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee7fca-c75a-4b5f-9c73-033a84cebeb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.972581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee7fca-c75a-4b5f-9c73-033a84cebeb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.973307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 794/1576] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:33.973959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579cbb3f-1636-4dc7-9bec-44499ad33ad6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:33.974994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee7fca-c75a-4b5f-9c73-033a84cebeb2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:33.983633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579cbb3f-1636-4dc7-9bec-44499ad33ad6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:33.988722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057b2de-5ced-4562-a358-fbf8b3aef613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:33.989131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057b2de-5ced-4562-a358-fbf8b3aef613 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:33.989462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057b2de-5ced-4562-a358-fbf8b3aef613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:33.989716 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057b2de-5ced-4562-a358-fbf8b3aef613 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:33.990398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057b2de-5ced-4562-a358-fbf8b3aef613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:33.992677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057b2de-5ced-4562-a358-fbf8b3aef613 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.004772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 783/1577] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:34.005635 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee7fca-c75a-4b5f-9c73-033a84cebeb2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.029976 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 795/1578] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:14:33 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:34.031543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57698c28-abbf-4834-a6fd-bacf1f108600 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.031898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57698c28-abbf-4834-a6fd-bacf1f108600 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.032181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57698c28-abbf-4834-a6fd-bacf1f108600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.051852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-57698c28-abbf-4834-a6fd-bacf1f108600 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:34.053049 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-57698c28-abbf-4834-a6fd-bacf1f108600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:34.070240 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee7fca-c75a-4b5f-9c73-033a84cebeb2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.114879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 796/1579] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:34.115734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57698c28-abbf-4834-a6fd-bacf1f108600 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.125412 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57698c28-abbf-4834-a6fd-bacf1f108600 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.130115 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a91428-7129-4055-a956-8e354c15721a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.130727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a91428-7129-4055-a956-8e354c15721a admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.131005 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a91428-7129-4055-a956-8e354c15721a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.131317 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a91428-7129-4055-a956-8e354c15721a admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.131657 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4d194-493a-4ee4-919c-9c2e4874d341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.131956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a91428-7129-4055-a956-8e354c15721a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.132279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4d194-493a-4ee4-919c-9c2e4874d341 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.132557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4d194-493a-4ee4-919c-9c2e4874d341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.132953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4d194-493a-4ee4-919c-9c2e4874d341 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.133584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4d194-493a-4ee4-919c-9c2e4874d341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.133880 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a91428-7129-4055-a956-8e354c15721a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.135541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4d194-493a-4ee4-919c-9c2e4874d341 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.156695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 797/1580] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:14:34 2022] POST /identity/v3/users => generated 291 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:34.157504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a91428-7129-4055-a956-8e354c15721a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.157938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 784/1581] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:14:34 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:34.161810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4d194-493a-4ee4-919c-9c2e4874d341 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.167498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a91428-7129-4055-a956-8e354c15721a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.168008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:34.172707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4d194-493a-4ee4-919c-9c2e4874d341 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.173221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ebce8-b417-4333-9538-84572aef278c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.173845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ebce8-b417-4333-9538-84572aef278c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.174124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ebce8-b417-4333-9538-84572aef278c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.174419 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ebce8-b417-4333-9538-84572aef278c admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.175034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ebce8-b417-4333-9538-84572aef278c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.177027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ebce8-b417-4333-9538-84572aef278c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.179009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e12dd-6680-4637-b819-1f01c2295acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.179724 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e12dd-6680-4637-b819-1f01c2295acc admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.180033 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e12dd-6680-4637-b819-1f01c2295acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.180303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e12dd-6680-4637-b819-1f01c2295acc admin admin] PATH_INFO: `/v3/projects/014fa4c673544a16b95c25d8af2cb70d/users/1ed72534f33f470993dc7e5406f84531/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.196783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e12dd-6680-4637-b819-1f01c2295acc admin admin] RBAC: Authorizing `identity:create_grant(project_id=014fa4c673544a16b95c25d8af2cb70d, user_id=1ed72534f33f470993dc7e5406f84531, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.200347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e12dd-6680-4637-b819-1f01c2295acc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.205438 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 798/1582] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:14:34 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:34.206253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ebce8-b417-4333-9538-84572aef278c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.217966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 785/1583] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:34 2022] PUT /identity/v3/projects/014fa4c673544a16b95c25d8af2cb70d/users/1ed72534f33f470993dc7e5406f84531/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:34.218768 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e12dd-6680-4637-b819-1f01c2295acc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.235740 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:34.279950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ebce8-b417-4333-9538-84572aef278c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.280155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e12dd-6680-4637-b819-1f01c2295acc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.284406 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fcc8f-d7d1-4066-a96c-2391281f0d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.284543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0c10d-2c88-4813-8c6c-f2d7a49109c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.285012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fcc8f-d7d1-4066-a96c-2391281f0d6e admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.285124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0c10d-2c88-4813-8c6c-f2d7a49109c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.285286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fcc8f-d7d1-4066-a96c-2391281f0d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.285397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0c10d-2c88-4813-8c6c-f2d7a49109c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.285554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fcc8f-d7d1-4066-a96c-2391281f0d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.285650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0c10d-2c88-4813-8c6c-f2d7a49109c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.286190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fcc8f-d7d1-4066-a96c-2391281f0d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.286294 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0c10d-2c88-4813-8c6c-f2d7a49109c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.287814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fcc8f-d7d1-4066-a96c-2391281f0d6e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.288198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0c10d-2c88-4813-8c6c-f2d7a49109c8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.298873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 799/1584] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:34.299634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fcc8f-d7d1-4066-a96c-2391281f0d6e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.310160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fcc8f-d7d1-4066-a96c-2391281f0d6e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.314750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 786/1585] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:14:34 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:34.314750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0c10d-2c88-4813-8c6c-f2d7a49109c8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.315286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a4453-575c-4ff3-9461-70de77f47d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.315886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a4453-575c-4ff3-9461-70de77f47d3a admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.316189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a4453-575c-4ff3-9461-70de77f47d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.316608 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a4453-575c-4ff3-9461-70de77f47d3a admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.317108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a4453-575c-4ff3-9461-70de77f47d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.319020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a4453-575c-4ff3-9461-70de77f47d3a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.340107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 800/1586] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:14:34 2022] POST /identity/v3/users => generated 292 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:34.341391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a4453-575c-4ff3-9461-70de77f47d3a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.352307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a4453-575c-4ff3-9461-70de77f47d3a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.357625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a1cdf-ca6b-41dc-a307-fa6dfd00afb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.358215 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1cdf-ca6b-41dc-a307-fa6dfd00afb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.358496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1cdf-ca6b-41dc-a307-fa6dfd00afb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.358766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1cdf-ca6b-41dc-a307-fa6dfd00afb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.359444 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a1cdf-ca6b-41dc-a307-fa6dfd00afb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.361743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a1cdf-ca6b-41dc-a307-fa6dfd00afb7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.369273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0c10d-2c88-4813-8c6c-f2d7a49109c8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.373316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 801/1587] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:34.373829 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e56640-ce94-4cd4-ae3b-479ca69d3658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.374138 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a1cdf-ca6b-41dc-a307-fa6dfd00afb7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.374515 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e56640-ce94-4cd4-ae3b-479ca69d3658 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.374802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e56640-ce94-4cd4-ae3b-479ca69d3658 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.375108 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e56640-ce94-4cd4-ae3b-479ca69d3658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.385001 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e56640-ce94-4cd4-ae3b-479ca69d3658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.385173 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a1cdf-ca6b-41dc-a307-fa6dfd00afb7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.386451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e56640-ce94-4cd4-ae3b-479ca69d3658 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.389919 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edc0a4-b487-4a57-bf1c-8d1ff484fa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.390489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edc0a4-b487-4a57-bf1c-8d1ff484fa6c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.390764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edc0a4-b487-4a57-bf1c-8d1ff484fa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.391033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edc0a4-b487-4a57-bf1c-8d1ff484fa6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.391678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edc0a4-b487-4a57-bf1c-8d1ff484fa6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.393235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edc0a4-b487-4a57-bf1c-8d1ff484fa6c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.396492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 787/1588] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:34.397374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e56640-ce94-4cd4-ae3b-479ca69d3658 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.404057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 802/1589] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:34.405640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2e0d9-8df8-4b39-bb22-fb50865d37fe None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.405931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2e0d9-8df8-4b39-bb22-fb50865d37fe None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.406209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2e0d9-8df8-4b39-bb22-fb50865d37fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.407614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e56640-ce94-4cd4-ae3b-479ca69d3658 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.413098 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59678953-9045-462c-b996-5fb55284ad4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.414021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59678953-9045-462c-b996-5fb55284ad4d admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.414565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59678953-9045-462c-b996-5fb55284ad4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.414565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59678953-9045-462c-b996-5fb55284ad4d admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.415214 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59678953-9045-462c-b996-5fb55284ad4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.417328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59678953-9045-462c-b996-5fb55284ad4d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.437575 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d1f2e0d9-8df8-4b39-bb22-fb50865d37fe None None] MFA Rules not processed for user `1ed72534f33f470993dc7e5406f84531`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:34.438925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d1f2e0d9-8df8-4b39-bb22-fb50865d37fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:34.457583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 788/1590] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:14:34 2022] POST /identity/v3/users => generated 286 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:34.457583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59678953-9045-462c-b996-5fb55284ad4d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.469274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59678953-9045-462c-b996-5fb55284ad4d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.477899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874492f2-800c-4887-99d2-1901e3fffbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.478479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874492f2-800c-4887-99d2-1901e3fffbb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.478765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874492f2-800c-4887-99d2-1901e3fffbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.479042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874492f2-800c-4887-99d2-1901e3fffbb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.479747 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874492f2-800c-4887-99d2-1901e3fffbb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.482236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874492f2-800c-4887-99d2-1901e3fffbb9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.501534 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 789/1591] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:34.501534 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874492f2-800c-4887-99d2-1901e3fffbb9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.514965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874492f2-800c-4887-99d2-1901e3fffbb9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.515510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 803/1592] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:34.516688 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2e0d9-8df8-4b39-bb22-fb50865d37fe None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.522609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ccc3b-e0df-4dad-b748-7dc0ee97e349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.523421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ccc3b-e0df-4dad-b748-7dc0ee97e349 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.523679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ccc3b-e0df-4dad-b748-7dc0ee97e349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.524510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ccc3b-e0df-4dad-b748-7dc0ee97e349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.525218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ccc3b-e0df-4dad-b748-7dc0ee97e349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.527104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ccc3b-e0df-4dad-b748-7dc0ee97e349 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.528621 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2e0d9-8df8-4b39-bb22-fb50865d37fe None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.535207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b93094-2104-4b7a-9855-dc208b555fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.535765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b93094-2104-4b7a-9855-dc208b555fbd admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.536250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b93094-2104-4b7a-9855-dc208b555fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.536482 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b93094-2104-4b7a-9855-dc208b555fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.537164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b93094-2104-4b7a-9855-dc208b555fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.538727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b93094-2104-4b7a-9855-dc208b555fbd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.543288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 790/1593] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:34.543288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ccc3b-e0df-4dad-b748-7dc0ee97e349 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.552960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ccc3b-e0df-4dad-b748-7dc0ee97e349 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.559254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ba152-f2dd-4d62-95f4-5a96355449e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.559879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 804/1594] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:34.560425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ba152-f2dd-4d62-95f4-5a96355449e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.560777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ba152-f2dd-4d62-95f4-5a96355449e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.560890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b93094-2104-4b7a-9855-dc208b555fbd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.561182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ba152-f2dd-4d62-95f4-5a96355449e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.561841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ba152-f2dd-4d62-95f4-5a96355449e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.563902 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ba152-f2dd-4d62-95f4-5a96355449e5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.570965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b93094-2104-4b7a-9855-dc208b555fbd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.578411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 791/1595] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:34.578705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e184e-7d17-4639-99ed-af8d1668998c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.578820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ba152-f2dd-4d62-95f4-5a96355449e5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.578900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e184e-7d17-4639-99ed-af8d1668998c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.578900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e184e-7d17-4639-99ed-af8d1668998c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.578900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e184e-7d17-4639-99ed-af8d1668998c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.578900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e184e-7d17-4639-99ed-af8d1668998c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.579920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e184e-7d17-4639-99ed-af8d1668998c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.593037 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 805/1596] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:34.593239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ba152-f2dd-4d62-95f4-5a96355449e5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.593893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e184e-7d17-4639-99ed-af8d1668998c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.598586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354233fd-b878-4be9-bfd4-0445c05d3665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.599342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354233fd-b878-4be9-bfd4-0445c05d3665 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.599719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354233fd-b878-4be9-bfd4-0445c05d3665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.600064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354233fd-b878-4be9-bfd4-0445c05d3665 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.600782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354233fd-b878-4be9-bfd4-0445c05d3665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.602649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354233fd-b878-4be9-bfd4-0445c05d3665 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.604425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e184e-7d17-4639-99ed-af8d1668998c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.612140 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4b81b-a83a-40d0-9ba1-65c021095e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.613694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4b81b-a83a-40d0-9ba1-65c021095e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.613694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4b81b-a83a-40d0-9ba1-65c021095e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.613694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4b81b-a83a-40d0-9ba1-65c021095e63 admin admin] PATH_INFO: `/v3/projects/6f6fd89d622647f49bd35dcaf8ceca55/users/c31bb28e38864eff99fdd6938b1b89a0/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.618017 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 792/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:34.619083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354233fd-b878-4be9-bfd4-0445c05d3665 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.632417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354233fd-b878-4be9-bfd4-0445c05d3665 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.632615 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4b81b-a83a-40d0-9ba1-65c021095e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6fd89d622647f49bd35dcaf8ceca55, user_id=c31bb28e38864eff99fdd6938b1b89a0, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.636433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4b81b-a83a-40d0-9ba1-65c021095e63 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.638461 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a71af-3046-47a3-8a37-a42ae73b16a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.639435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a71af-3046-47a3-8a37-a42ae73b16a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.640187 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a71af-3046-47a3-8a37-a42ae73b16a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.640537 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a71af-3046-47a3-8a37-a42ae73b16a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.641436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a71af-3046-47a3-8a37-a42ae73b16a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.644002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a71af-3046-47a3-8a37-a42ae73b16a4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.656524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 806/1598] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:34 2022] PUT /identity/v3/projects/6f6fd89d622647f49bd35dcaf8ceca55/users/c31bb28e38864eff99fdd6938b1b89a0/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:34.656524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4b81b-a83a-40d0-9ba1-65c021095e63 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.656836 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:34.679540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:34.682940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 793/1599] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:14:34 2022] POST /identity/v3/projects => generated 355 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:34.684092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a71af-3046-47a3-8a37-a42ae73b16a4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.736783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4b81b-a83a-40d0-9ba1-65c021095e63 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.754594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a71af-3046-47a3-8a37-a42ae73b16a4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.759549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633683e-ce44-41a6-a9c2-393bfd35f0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.760237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633683e-ce44-41a6-a9c2-393bfd35f0da admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.760584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633683e-ce44-41a6-a9c2-393bfd35f0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.760924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633683e-ce44-41a6-a9c2-393bfd35f0da admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.761674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633683e-ce44-41a6-a9c2-393bfd35f0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.765080 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633683e-ce44-41a6-a9c2-393bfd35f0da admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.779132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 794/1600] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:34.780564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633683e-ce44-41a6-a9c2-393bfd35f0da admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.792145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633683e-ce44-41a6-a9c2-393bfd35f0da admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.797144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ca7b6-d630-4982-88fd-6fc8e0f1f03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.797799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ca7b6-d630-4982-88fd-6fc8e0f1f03f admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.798074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ca7b6-d630-4982-88fd-6fc8e0f1f03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.798343 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ca7b6-d630-4982-88fd-6fc8e0f1f03f admin admin] PATH_INFO: `/v3/projects/f6e2f1f36448419ea1d12bbc8e4d1100/users/2a98941032454d0f969f3cde046bbaf0/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.804251 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43998a3-cc05-4793-a0c8-fce845ae6b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.805763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43998a3-cc05-4793-a0c8-fce845ae6b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.806214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43998a3-cc05-4793-a0c8-fce845ae6b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.806818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43998a3-cc05-4793-a0c8-fce845ae6b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.808563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43998a3-cc05-4793-a0c8-fce845ae6b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.810764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43998a3-cc05-4793-a0c8-fce845ae6b1b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.826987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ca7b6-d630-4982-88fd-6fc8e0f1f03f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e2f1f36448419ea1d12bbc8e4d1100, user_id=2a98941032454d0f969f3cde046bbaf0, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.827771 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 807/1601] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:34.827975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ca7b6-d630-4982-88fd-6fc8e0f1f03f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.829671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43998a3-cc05-4793-a0c8-fce845ae6b1b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.842208 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43998a3-cc05-4793-a0c8-fce845ae6b1b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.849535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 795/1602] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:34 2022] PUT /identity/v3/projects/f6e2f1f36448419ea1d12bbc8e4d1100/users/2a98941032454d0f969f3cde046bbaf0/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:34.863386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ca7b6-d630-4982-88fd-6fc8e0f1f03f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.928766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c273396-0a0d-4475-8c6f-8c5bf5d1e6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.930168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c273396-0a0d-4475-8c6f-8c5bf5d1e6d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.930457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c273396-0a0d-4475-8c6f-8c5bf5d1e6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.931177 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c273396-0a0d-4475-8c6f-8c5bf5d1e6d0 admin admin] PATH_INFO: `/v3/projects/6b24b42ca93941aa8e70f5fa10303f4d/users/dcb88983a8e1439588644b71170d7362/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.944500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ca7b6-d630-4982-88fd-6fc8e0f1f03f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:34.948109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c273396-0a0d-4475-8c6f-8c5bf5d1e6d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b24b42ca93941aa8e70f5fa10303f4d, user_id=dcb88983a8e1439588644b71170d7362, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.950489 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7e8b3-96e0-4ded-bc9b-bdd55d645680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:34.951615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7e8b3-96e0-4ded-bc9b-bdd55d645680 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.951834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c273396-0a0d-4475-8c6f-8c5bf5d1e6d0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.952457 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7e8b3-96e0-4ded-bc9b-bdd55d645680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.952932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7e8b3-96e0-4ded-bc9b-bdd55d645680 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.953700 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7e8b3-96e0-4ded-bc9b-bdd55d645680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:34.956076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7e8b3-96e0-4ded-bc9b-bdd55d645680 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:34.970620 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 796/1603] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:34.970620 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e4aa2-89e9-45f1-85de-98999aad89aa None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:34.970620 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e4aa2-89e9-45f1-85de-98999aad89aa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:34.970620 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e4aa2-89e9-45f1-85de-98999aad89aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:34.975167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 808/1604] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:34 2022] PUT /identity/v3/projects/6b24b42ca93941aa8e70f5fa10303f4d/users/dcb88983a8e1439588644b71170d7362/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:34.975167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c273396-0a0d-4475-8c6f-8c5bf5d1e6d0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:34.992695 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0c8e4aa2-89e9-45f1-85de-98999aad89aa None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:34.994168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0c8e4aa2-89e9-45f1-85de-98999aad89aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:35.033000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:35.052307 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:35.055517 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c273396-0a0d-4475-8c6f-8c5bf5d1e6d0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:35.060770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a266904-9571-4154-9b0f-80f71f44c2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:35.061456 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a266904-9571-4154-9b0f-80f71f44c2c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.061749 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a266904-9571-4154-9b0f-80f71f44c2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.062019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a266904-9571-4154-9b0f-80f71f44c2c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.062649 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a266904-9571-4154-9b0f-80f71f44c2c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:35.064214 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 797/1605] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:35.064929 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e4aa2-89e9-45f1-85de-98999aad89aa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:35.068083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a266904-9571-4154-9b0f-80f71f44c2c2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:35.075287 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e4aa2-89e9-45f1-85de-98999aad89aa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:35.080880 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de636af1-e2b0-4727-bbc4-e74e3fd768d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:35.081759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de636af1-e2b0-4727-bbc4-e74e3fd768d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.082596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de636af1-e2b0-4727-bbc4-e74e3fd768d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.082756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de636af1-e2b0-4727-bbc4-e74e3fd768d8 admin admin] PATH_INFO: `/v3/projects/995f9843e1994072b79c1f0b38c96140/users/021f677dcf064541835e9a68efe5b145/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.096540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 809/1606] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:14:34 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:35.097558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a266904-9571-4154-9b0f-80f71f44c2c2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:35.104799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de636af1-e2b0-4727-bbc4-e74e3fd768d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=995f9843e1994072b79c1f0b38c96140, user_id=021f677dcf064541835e9a68efe5b145, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:35.109659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a266904-9571-4154-9b0f-80f71f44c2c2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:35.110755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de636af1-e2b0-4727-bbc4-e74e3fd768d8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:35.114596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361dd22-a129-48e7-adea-66e1c6f5126b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:35.115838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361dd22-a129-48e7-adea-66e1c6f5126b admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.115838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361dd22-a129-48e7-adea-66e1c6f5126b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.116026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361dd22-a129-48e7-adea-66e1c6f5126b admin admin] PATH_INFO: `/v3/projects/1eedefa6b656409f902d158e77d94aa0/users/21700a3c2ff24fc6a3bb5db8bf283f5c/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.129681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 798/1607] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:35 2022] PUT /identity/v3/projects/995f9843e1994072b79c1f0b38c96140/users/021f677dcf064541835e9a68efe5b145/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:35.131178 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0054b3-c1a1-48e4-bcbb-efd31f9f5749 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.131779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0054b3-c1a1-48e4-bcbb-efd31f9f5749 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.131779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0054b3-c1a1-48e4-bcbb-efd31f9f5749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.135326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361dd22-a129-48e7-adea-66e1c6f5126b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eedefa6b656409f902d158e77d94aa0, user_id=21700a3c2ff24fc6a3bb5db8bf283f5c, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:35.140641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361dd22-a129-48e7-adea-66e1c6f5126b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:35.156547 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 810/1608] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:35 2022] PUT /identity/v3/projects/1eedefa6b656409f902d158e77d94aa0/users/21700a3c2ff24fc6a3bb5db8bf283f5c/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:35.157969 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361dd22-a129-48e7-adea-66e1c6f5126b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:35.168346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6c0054b3-c1a1-48e4-bcbb-efd31f9f5749 None None] MFA Rules not processed for user `2a98941032454d0f969f3cde046bbaf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:35.170498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6c0054b3-c1a1-48e4-bcbb-efd31f9f5749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:35.238266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361dd22-a129-48e7-adea-66e1c6f5126b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:35.245224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaac4eb-0a62-4d86-a520-6d1b5935822e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:35.247341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaac4eb-0a62-4d86-a520-6d1b5935822e admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.247697 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaac4eb-0a62-4d86-a520-6d1b5935822e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.248062 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaac4eb-0a62-4d86-a520-6d1b5935822e admin admin] PATH_INFO: `/v3/projects/17775293d617489c8d24f758ad5142ce/users/fd1a7ec2da294f3a92b061a87b306a19/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.273899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaac4eb-0a62-4d86-a520-6d1b5935822e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17775293d617489c8d24f758ad5142ce, user_id=fd1a7ec2da294f3a92b061a87b306a19, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:35.274989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 799/1609] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:35.276463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0dda8-d608-4423-abdc-722c9d442575 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.277150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0dda8-d608-4423-abdc-722c9d442575 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.278163 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0dda8-d608-4423-abdc-722c9d442575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.279125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaac4eb-0a62-4d86-a520-6d1b5935822e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:35.299110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 811/1610] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:35 2022] PUT /identity/v3/projects/17775293d617489c8d24f758ad5142ce/users/fd1a7ec2da294f3a92b061a87b306a19/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:35.299110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaac4eb-0a62-4d86-a520-6d1b5935822e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:35.313786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c3b0dda8-d608-4423-abdc-722c9d442575 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:35.315175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c3b0dda8-d608-4423-abdc-722c9d442575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:35.383491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaac4eb-0a62-4d86-a520-6d1b5935822e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:35.388314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958f3f0-27d8-482d-8b85-d438a3c62291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:35.389397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958f3f0-27d8-482d-8b85-d438a3c62291 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.389845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958f3f0-27d8-482d-8b85-d438a3c62291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.390396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958f3f0-27d8-482d-8b85-d438a3c62291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.392855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 800/1611] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:35.393790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0dda8-d608-4423-abdc-722c9d442575 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:35.405273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0dda8-d608-4423-abdc-722c9d442575 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:35.407762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958f3f0-27d8-482d-8b85-d438a3c62291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:35.409283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958f3f0-27d8-482d-8b85-d438a3c62291 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:35.410730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020bcb5-ca84-4b5a-91b0-3e72e3f9116f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:35.412363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020bcb5-ca84-4b5a-91b0-3e72e3f9116f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.413012 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020bcb5-ca84-4b5a-91b0-3e72e3f9116f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.413318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020bcb5-ca84-4b5a-91b0-3e72e3f9116f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.414238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020bcb5-ca84-4b5a-91b0-3e72e3f9116f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:35.416058 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020bcb5-ca84-4b5a-91b0-3e72e3f9116f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:35.425554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 812/1612] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:35.427683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 801/1613] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:35.428042 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0fe96-a322-42a7-ad23-9b848b719bfc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.428459 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0fe96-a322-42a7-ad23-9b848b719bfc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.428800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0fe96-a322-42a7-ad23-9b848b719bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.429381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d35b1b-ec9a-4883-9b96-cfa8a12139fe None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.429671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d35b1b-ec9a-4883-9b96-cfa8a12139fe None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.429949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d35b1b-ec9a-4883-9b96-cfa8a12139fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.453248 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-88d35b1b-ec9a-4883-9b96-cfa8a12139fe None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:35.454640 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-88d35b1b-ec9a-4883-9b96-cfa8a12139fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:35.467196 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 802/1614] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:35.469427 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-91d0fe96-a322-42a7-ad23-9b848b719bfc None None] MFA Rules not processed for user `021f677dcf064541835e9a68efe5b145`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:35.470343 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d35b1b-ec9a-4883-9b96-cfa8a12139fe None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:35.472062 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-91d0fe96-a322-42a7-ad23-9b848b719bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:35.487476 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d35b1b-ec9a-4883-9b96-cfa8a12139fe None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:35.497398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4c824-83a8-4872-9fcf-a03fb0172346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:35.498779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4c824-83a8-4872-9fcf-a03fb0172346 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.499334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4c824-83a8-4872-9fcf-a03fb0172346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.499953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4c824-83a8-4872-9fcf-a03fb0172346 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.500808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4c824-83a8-4872-9fcf-a03fb0172346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:35.503403 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4c824-83a8-4872-9fcf-a03fb0172346 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:35.516497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:35.554938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 803/1615] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:14:35 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:35.556758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeea40a-6ee4-4d86-9986-f5e95106180a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.557102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeea40a-6ee4-4d86-9986-f5e95106180a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.557386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeea40a-6ee4-4d86-9986-f5e95106180a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.601009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ceeea40a-6ee4-4d86-9986-f5e95106180a None None] MFA Rules not processed for user `fd1a7ec2da294f3a92b061a87b306a19`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:35.601733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ceeea40a-6ee4-4d86-9986-f5e95106180a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:35.604301 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 813/1616] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:35.605485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0fe96-a322-42a7-ad23-9b848b719bfc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:35.624789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-91d0fe96-a322-42a7-ad23-9b848b719bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:35.628122 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:35.660650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0fe96-a322-42a7-ad23-9b848b719bfc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:35.732922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 804/1617] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:35.734003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeea40a-6ee4-4d86-9986-f5e95106180a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:35.745705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef166e-857c-48b4-bc3f-821dbcd5b921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:35.747010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef166e-857c-48b4-bc3f-821dbcd5b921 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.747421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef166e-857c-48b4-bc3f-821dbcd5b921 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.747708 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef166e-857c-48b4-bc3f-821dbcd5b921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.836585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeea40a-6ee4-4d86-9986-f5e95106180a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:35.841808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef166e-857c-48b4-bc3f-821dbcd5b921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:35.843497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058fcf0-58b8-413c-9f62-0f092e917bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:35.843619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef166e-857c-48b4-bc3f-821dbcd5b921 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:35.844095 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058fcf0-58b8-413c-9f62-0f092e917bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.844506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058fcf0-58b8-413c-9f62-0f092e917bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.844694 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058fcf0-58b8-413c-9f62-0f092e917bdc admin admin] PATH_INFO: `/v3/projects/ae3e51af6f6644cc84a7872c7f926fcc/users/4a6e05c2a7fd4bad81d46ff6fe00cae0/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.854946 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 814/1618] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:35.856449 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef166e-857c-48b4-bc3f-821dbcd5b921 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:35.870892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058fcf0-58b8-413c-9f62-0f092e917bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3e51af6f6644cc84a7872c7f926fcc, user_id=4a6e05c2a7fd4bad81d46ff6fe00cae0, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:35.872929 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef166e-857c-48b4-bc3f-821dbcd5b921 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:35.875726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058fcf0-58b8-413c-9f62-0f092e917bdc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:35.877494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fce6a7-5095-45f8-ba90-76cf69aa2d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:35.878846 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fce6a7-5095-45f8-ba90-76cf69aa2d2d service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.879989 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fce6a7-5095-45f8-ba90-76cf69aa2d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.880121 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fce6a7-5095-45f8-ba90-76cf69aa2d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.900332 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 805/1619] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:35 2022] PUT /identity/v3/projects/ae3e51af6f6644cc84a7872c7f926fcc/users/4a6e05c2a7fd4bad81d46ff6fe00cae0/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:35.900524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fce6a7-5095-45f8-ba90-76cf69aa2d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:35.900646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058fcf0-58b8-413c-9f62-0f092e917bdc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:35.900749 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fce6a7-5095-45f8-ba90-76cf69aa2d2d service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:35.920170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 815/1620] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:35.922390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc832d9b-f15a-4339-a0f0-ad9ff1fb0ba2 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:35.922804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc832d9b-f15a-4339-a0f0-ad9ff1fb0ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:35.923101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc832d9b-f15a-4339-a0f0-ad9ff1fb0ba2 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:35.924844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 816/1621] 10.222.0.17 () {54 vars in 962 bytes} [Fri Sep 9 13:14:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:14:35.925555 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc832d9b-f15a-4339-a0f0-ad9ff1fb0ba2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:36.050671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc832d9b-f15a-4339-a0f0-ad9ff1fb0ba2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:36.057836 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058fcf0-58b8-413c-9f62-0f092e917bdc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:36.058001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c7d45-eccd-4194-b3b6-91cf9aaee5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:36.058547 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c7d45-eccd-4194-b3b6-91cf9aaee5bc admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:36.059891 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c7d45-eccd-4194-b3b6-91cf9aaee5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:36.060709 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c7d45-eccd-4194-b3b6-91cf9aaee5bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:36.061769 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c7d45-eccd-4194-b3b6-91cf9aaee5bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:36.064108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c7d45-eccd-4194-b3b6-91cf9aaee5bc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:36.069307 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57d2bb-4e8f-43a7-813f-891b743fb62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:36.069958 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57d2bb-4e8f-43a7-813f-891b743fb62b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:36.070238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57d2bb-4e8f-43a7-813f-891b743fb62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:36.070516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57d2bb-4e8f-43a7-813f-891b743fb62b admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:36.071199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57d2bb-4e8f-43a7-813f-891b743fb62b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:36.076998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:36.078877 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57d2bb-4e8f-43a7-813f-891b743fb62b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:36.120454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 817/1622] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:14:35 2022] POST /identity/v3/projects => generated 371 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:36.122329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c7d45-eccd-4194-b3b6-91cf9aaee5bc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:36.130427 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 806/1623] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:14:35 2022] POST /identity/v3/users => generated 295 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:36.131722 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9735848-17d9-463b-bb5b-0e6450029c48 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:36.132010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9735848-17d9-463b-bb5b-0e6450029c48 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:36.134689 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9735848-17d9-463b-bb5b-0e6450029c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:36.231522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a9735848-17d9-463b-bb5b-0e6450029c48 None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:36.234989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a9735848-17d9-463b-bb5b-0e6450029c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:36.241803 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c7d45-eccd-4194-b3b6-91cf9aaee5bc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:36.249559 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd83d53-5f21-44fc-b820-271d46a0079c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:36.250286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd83d53-5f21-44fc-b820-271d46a0079c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:36.250566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd83d53-5f21-44fc-b820-271d46a0079c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:36.250856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd83d53-5f21-44fc-b820-271d46a0079c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:36.251694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd83d53-5f21-44fc-b820-271d46a0079c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:36.254594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd83d53-5f21-44fc-b820-271d46a0079c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:36.260060 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:36.270051 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 818/1624] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:36.271083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd83d53-5f21-44fc-b820-271d46a0079c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:36.285702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd83d53-5f21-44fc-b820-271d46a0079c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:36.296337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19f441-c186-4376-a9a1-6c2e11b3cf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:36.297021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19f441-c186-4376-a9a1-6c2e11b3cf42 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:36.297302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19f441-c186-4376-a9a1-6c2e11b3cf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:36.297572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19f441-c186-4376-a9a1-6c2e11b3cf42 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:36.298558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19f441-c186-4376-a9a1-6c2e11b3cf42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:36.301374 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19f441-c186-4376-a9a1-6c2e11b3cf42 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:36.348013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 819/1625] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:14:36 2022] POST /identity/v3/users => generated 292 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:14:36.348013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19f441-c186-4376-a9a1-6c2e11b3cf42 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:36.363740 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19f441-c186-4376-a9a1-6c2e11b3cf42 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:36.370539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ce58b-f091-4d21-b962-855dfac89a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:36.371411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ce58b-f091-4d21-b962-855dfac89a3a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:36.372478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ce58b-f091-4d21-b962-855dfac89a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:36.372803 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ce58b-f091-4d21-b962-855dfac89a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:36.373468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ce58b-f091-4d21-b962-855dfac89a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:36.375279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ce58b-f091-4d21-b962-855dfac89a3a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:36.385869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 807/1626] 10.222.0.17 () {58 vars in 1059 bytes} [Fri Sep 9 13:14:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:36.387231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9735848-17d9-463b-bb5b-0e6450029c48 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:36.405356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 820/1627] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:36.408245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9735848-17d9-463b-bb5b-0e6450029c48 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:36.409020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ce58b-f091-4d21-b962-855dfac89a3a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:36.422021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e4297-f9e7-4bd7-bb61-950d0340b285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:36.422667 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e4297-f9e7-4bd7-bb61-950d0340b285 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:36.422961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e4297-f9e7-4bd7-bb61-950d0340b285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:36.423624 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e4297-f9e7-4bd7-bb61-950d0340b285 admin admin] PATH_INFO: `/v3/projects/ae3e51af6f6644cc84a7872c7f926fcc/users/4a6e05c2a7fd4bad81d46ff6fe00cae0/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:36.428140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e4297-f9e7-4bd7-bb61-950d0340b285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3e51af6f6644cc84a7872c7f926fcc, user_id=4a6e05c2a7fd4bad81d46ff6fe00cae0, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:36.431188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e4297-f9e7-4bd7-bb61-950d0340b285 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:36.435970 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ce58b-f091-4d21-b962-855dfac89a3a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:36.447007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5d5f4-b22f-4a14-a1df-efa60851c8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:36.448686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5d5f4-b22f-4a14-a1df-efa60851c8ed admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:36.449422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5d5f4-b22f-4a14-a1df-efa60851c8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:36.449959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5d5f4-b22f-4a14-a1df-efa60851c8ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:36.460626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 808/1628] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:36 2022] PUT /identity/v3/projects/ae3e51af6f6644cc84a7872c7f926fcc/users/4a6e05c2a7fd4bad81d46ff6fe00cae0/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:36.460626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e4297-f9e7-4bd7-bb61-950d0340b285 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:36.471465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5d5f4-b22f-4a14-a1df-efa60851c8ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:36.474819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5d5f4-b22f-4a14-a1df-efa60851c8ed admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:36.502450 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 821/1629] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:36.507813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5d5f4-b22f-4a14-a1df-efa60851c8ed admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:36.520868 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5d5f4-b22f-4a14-a1df-efa60851c8ed admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:36.684259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e4297-f9e7-4bd7-bb61-950d0340b285 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:36.703841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcd3fb-d91c-4af0-94ec-82ec1e5798cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:36.704715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcd3fb-d91c-4af0-94ec-82ec1e5798cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:36.705100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcd3fb-d91c-4af0-94ec-82ec1e5798cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:36.705479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcd3fb-d91c-4af0-94ec-82ec1e5798cf admin admin] PATH_INFO: `/v3/projects/ff462ec87a0541d8997d9ea9fd44aab3/users/3a0d5f56cd544425bd0f96e5063d206c/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:36.714378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5d522-7fa8-45e2-aa5f-3088007bb7a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:36.715017 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5d522-7fa8-45e2-aa5f-3088007bb7a3 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:36.719766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5d522-7fa8-45e2-aa5f-3088007bb7a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:36.720131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5d522-7fa8-45e2-aa5f-3088007bb7a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:36.744001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcd3fb-d91c-4af0-94ec-82ec1e5798cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff462ec87a0541d8997d9ea9fd44aab3, user_id=3a0d5f56cd544425bd0f96e5063d206c, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:36.753387 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcd3fb-d91c-4af0-94ec-82ec1e5798cf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:36.782339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 822/1630] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:36 2022] PUT /identity/v3/projects/ff462ec87a0541d8997d9ea9fd44aab3/users/3a0d5f56cd544425bd0f96e5063d206c/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:36.782339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4528c-34c9-4f37-ab31-eb91034423bc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:36.782339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4528c-34c9-4f37-ab31-eb91034423bc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:36.783196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4528c-34c9-4f37-ab31-eb91034423bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:36.832397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5cf4528c-34c9-4f37-ab31-eb91034423bc None None] MFA Rules not processed for user `4a6e05c2a7fd4bad81d46ff6fe00cae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:36.833659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5cf4528c-34c9-4f37-ab31-eb91034423bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:36.848334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:36.877414 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5d522-7fa8-45e2-aa5f-3088007bb7a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:36.879081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5d522-7fa8-45e2-aa5f-3088007bb7a3 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:36.900590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 809/1631] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:36.901453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5d522-7fa8-45e2-aa5f-3088007bb7a3 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:36.925188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5d522-7fa8-45e2-aa5f-3088007bb7a3 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:36.972715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 823/1632] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:36.973702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4528c-34c9-4f37-ab31-eb91034423bc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:36.995716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:37.074049 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4528c-34c9-4f37-ab31-eb91034423bc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:37.087332 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf98ddb-394f-4ae0-85f1-e2d4b6c31e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:37.087964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b8fb6-32a9-4eaa-b8f9-181f158ea149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:37.088120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf98ddb-394f-4ae0-85f1-e2d4b6c31e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:37.088290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf98ddb-394f-4ae0-85f1-e2d4b6c31e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:37.088568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf98ddb-394f-4ae0-85f1-e2d4b6c31e92 admin admin] PATH_INFO: `/v3/projects/745bdeeed0ab43bbbdc84f0bf212826c/users/41db780aa8954fbcbd5f8cdf820707b8/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:37.089355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b8fb6-32a9-4eaa-b8f9-181f158ea149 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:37.089796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b8fb6-32a9-4eaa-b8f9-181f158ea149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:37.090469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b8fb6-32a9-4eaa-b8f9-181f158ea149 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:37.091499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b8fb6-32a9-4eaa-b8f9-181f158ea149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:37.093983 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b8fb6-32a9-4eaa-b8f9-181f158ea149 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:37.115977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 824/1633] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:37.117336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b8fb6-32a9-4eaa-b8f9-181f158ea149 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:37.152919 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf98ddb-394f-4ae0-85f1-e2d4b6c31e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=745bdeeed0ab43bbbdc84f0bf212826c, user_id=41db780aa8954fbcbd5f8cdf820707b8, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:37.161142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf98ddb-394f-4ae0-85f1-e2d4b6c31e92 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:37.193403 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 810/1634] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:36 2022] PUT /identity/v3/projects/745bdeeed0ab43bbbdc84f0bf212826c/users/41db780aa8954fbcbd5f8cdf820707b8/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:37.194467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf98ddb-394f-4ae0-85f1-e2d4b6c31e92 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:37.246139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b8fb6-32a9-4eaa-b8f9-181f158ea149 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:37.370830 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf98ddb-394f-4ae0-85f1-e2d4b6c31e92 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:37.384924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f804fb5-9c94-4144-9526-ddb1635ea15f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:37.385605 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f804fb5-9c94-4144-9526-ddb1635ea15f service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:37.385884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f804fb5-9c94-4144-9526-ddb1635ea15f service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:37.386179 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f804fb5-9c94-4144-9526-ddb1635ea15f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:37.395145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f86d8-b89d-4900-aacb-2af22709ae1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:37.395867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f86d8-b89d-4900-aacb-2af22709ae1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:37.396147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f86d8-b89d-4900-aacb-2af22709ae1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:37.396418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f86d8-b89d-4900-aacb-2af22709ae1e admin admin] PATH_INFO: `/v3/projects/ff462ec87a0541d8997d9ea9fd44aab3/users/3a0d5f56cd544425bd0f96e5063d206c/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:37.401639 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f86d8-b89d-4900-aacb-2af22709ae1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff462ec87a0541d8997d9ea9fd44aab3, user_id=3a0d5f56cd544425bd0f96e5063d206c, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:37.408763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f86d8-b89d-4900-aacb-2af22709ae1e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:37.428216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:37.440709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 811/1635] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:37 2022] PUT /identity/v3/projects/ff462ec87a0541d8997d9ea9fd44aab3/users/3a0d5f56cd544425bd0f96e5063d206c/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:37.441756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f86d8-b89d-4900-aacb-2af22709ae1e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:37.546441 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f804fb5-9c94-4144-9526-ddb1635ea15f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:37.549762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f804fb5-9c94-4144-9526-ddb1635ea15f service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:37.575025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 825/1636] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:37.588155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebed02-3679-4d89-885c-8992097c5f9b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:37.588497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebed02-3679-4d89-885c-8992097c5f9b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:37.588801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebed02-3679-4d89-885c-8992097c5f9b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:37.601751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 826/1637] 10.222.0.17 () {54 vars in 912 bytes} [Fri Sep 9 13:14:37 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:14:37.605679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65cf87-6617-4266-840a-5d2f79eb9a16 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:37.610862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65cf87-6617-4266-840a-5d2f79eb9a16 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:37.610862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65cf87-6617-4266-840a-5d2f79eb9a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:37.611314 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f86d8-b89d-4900-aacb-2af22709ae1e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:37.623363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08854153-f45a-4f63-a2b9-0af658fa05f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:37.624154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08854153-f45a-4f63-a2b9-0af658fa05f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:37.624547 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08854153-f45a-4f63-a2b9-0af658fa05f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:37.625093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08854153-f45a-4f63-a2b9-0af658fa05f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:37.626603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08854153-f45a-4f63-a2b9-0af658fa05f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:37.630289 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08854153-f45a-4f63-a2b9-0af658fa05f4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:37.659609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 812/1638] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:14:37.661232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ea206-e57b-4180-a5ad-210af513c9ba None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:37.661628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ea206-e57b-4180-a5ad-210af513c9ba None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:37.662039 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ea206-e57b-4180-a5ad-210af513c9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:37.701914 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5d65cf87-6617-4266-840a-5d2f79eb9a16 None None] MFA Rules not processed for user `3a0d5f56cd544425bd0f96e5063d206c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:37.703667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5d65cf87-6617-4266-840a-5d2f79eb9a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:37.717206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-289ea206-e57b-4180-a5ad-210af513c9ba None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:37.720488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-289ea206-e57b-4180-a5ad-210af513c9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:37.820704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:37.880266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 827/1639] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:37.881214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65cf87-6617-4266-840a-5d2f79eb9a16 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:37.887319 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 813/1640] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:14:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:37.888297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ea206-e57b-4180-a5ad-210af513c9ba None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:37.911968 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65cf87-6617-4266-840a-5d2f79eb9a16 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:37.921378 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda507d6-798f-4788-a618-31f29a1fed94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:37.922076 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda507d6-798f-4788-a618-31f29a1fed94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:37.922360 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda507d6-798f-4788-a618-31f29a1fed94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:37.922641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda507d6-798f-4788-a618-31f29a1fed94 admin admin] PATH_INFO: `/v3/projects/745bdeeed0ab43bbbdc84f0bf212826c/users/41db780aa8954fbcbd5f8cdf820707b8/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:37.927467 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda507d6-798f-4788-a618-31f29a1fed94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=745bdeeed0ab43bbbdc84f0bf212826c, user_id=41db780aa8954fbcbd5f8cdf820707b8, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:37.935791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda507d6-798f-4788-a618-31f29a1fed94 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:37.967944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 828/1641] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:14:37 2022] PUT /identity/v3/projects/745bdeeed0ab43bbbdc84f0bf212826c/users/41db780aa8954fbcbd5f8cdf820707b8/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:37.969158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda507d6-798f-4788-a618-31f29a1fed94 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:37.991610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ea206-e57b-4180-a5ad-210af513c9ba None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:38.119093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a934acc-79ef-448b-a7e9-34d6ccb9fc3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:38.119732 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a934acc-79ef-448b-a7e9-34d6ccb9fc3a service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:38.120014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a934acc-79ef-448b-a7e9-34d6ccb9fc3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:38.120283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a934acc-79ef-448b-a7e9-34d6ccb9fc3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:38.124576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda507d6-798f-4788-a618-31f29a1fed94 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:38.130528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b9e99-eda4-4c39-b112-91c5fb074efa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:38.131961 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b9e99-eda4-4c39-b112-91c5fb074efa service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:38.132384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b9e99-eda4-4c39-b112-91c5fb074efa service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:38.132904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b9e99-eda4-4c39-b112-91c5fb074efa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:38.226502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:38.244175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a934acc-79ef-448b-a7e9-34d6ccb9fc3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:38.246147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a934acc-79ef-448b-a7e9-34d6ccb9fc3a service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:38.258745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 814/1642] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:38.260582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53177547-ef67-489b-b14f-666c1a3fb3cb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:38.260875 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53177547-ef67-489b-b14f-666c1a3fb3cb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:38.261153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53177547-ef67-489b-b14f-666c1a3fb3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:38.298709 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b9e99-eda4-4c39-b112-91c5fb074efa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:38.301634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b9e99-eda4-4c39-b112-91c5fb074efa service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:38.302977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-53177547-ef67-489b-b14f-666c1a3fb3cb None None] MFA Rules not processed for user `41db780aa8954fbcbd5f8cdf820707b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:38.305026 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-53177547-ef67-489b-b14f-666c1a3fb3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:38.333743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 829/1643] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:38.354223 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:38.416207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b9e99-eda4-4c39-b112-91c5fb074efa service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:38.439853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b9e99-eda4-4c39-b112-91c5fb074efa service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:38.454738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd30f25-7f4a-485c-af49-ef0d51aba103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:38.455378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 815/1644] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:38.457433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd30f25-7f4a-485c-af49-ef0d51aba103 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:38.457690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd30f25-7f4a-485c-af49-ef0d51aba103 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:38.458485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd30f25-7f4a-485c-af49-ef0d51aba103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:38.603781 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd30f25-7f4a-485c-af49-ef0d51aba103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:38.605462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd30f25-7f4a-485c-af49-ef0d51aba103 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:38.626848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 830/1645] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:14:38.812806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53177547-ef67-489b-b14f-666c1a3fb3cb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:38.831524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53177547-ef67-489b-b14f-666c1a3fb3cb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:38.839838 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cfbca-441c-4bba-baa9-83b9c1870b79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:38.841361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cfbca-441c-4bba-baa9-83b9c1870b79 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:38.841818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cfbca-441c-4bba-baa9-83b9c1870b79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:38.842096 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cfbca-441c-4bba-baa9-83b9c1870b79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:38.864839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cfbca-441c-4bba-baa9-83b9c1870b79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:38.866415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cfbca-441c-4bba-baa9-83b9c1870b79 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:38.889641 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 816/1646] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:39.073428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd30f25-7f4a-485c-af49-ef0d51aba103 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:39.088478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd30f25-7f4a-485c-af49-ef0d51aba103 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:39.101078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7dff2-9eae-481b-b8f8-9c3093c67445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:39.103287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7dff2-9eae-481b-b8f8-9c3093c67445 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:39.104140 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7dff2-9eae-481b-b8f8-9c3093c67445 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:39.104464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7dff2-9eae-481b-b8f8-9c3093c67445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:39.259853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b7dff2-9eae-481b-b8f8-9c3093c67445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:39.261657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b7dff2-9eae-481b-b8f8-9c3093c67445 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:39.280106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 831/1647] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:39.723748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cfbca-441c-4bba-baa9-83b9c1870b79 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:39.735791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cfbca-441c-4bba-baa9-83b9c1870b79 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:39.746044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7c738-37a4-454c-a0d1-9a75b26e9667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:39.746177 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7c738-37a4-454c-a0d1-9a75b26e9667 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:39.747072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7c738-37a4-454c-a0d1-9a75b26e9667 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:39.748205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7c738-37a4-454c-a0d1-9a75b26e9667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:39.921922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7c738-37a4-454c-a0d1-9a75b26e9667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:39.956564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7c738-37a4-454c-a0d1-9a75b26e9667 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:39.970811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 817/1648] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:40.313689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64964a36-a2e8-4689-a555-effeb33bcbd6 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:40.314292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64964a36-a2e8-4689-a555-effeb33bcbd6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:40.314292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64964a36-a2e8-4689-a555-effeb33bcbd6 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:40.316052 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 832/1649] 10.222.0.17 () {54 vars in 962 bytes} [Fri Sep 9 13:14:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:14:40.328269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0fac5-876a-4a14-a06f-1db0c0b7d3ec None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:40.328552 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0fac5-876a-4a14-a06f-1db0c0b7d3ec None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:40.328825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0fac5-876a-4a14-a06f-1db0c0b7d3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:40.376078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f6d0fac5-876a-4a14-a06f-1db0c0b7d3ec None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:40.377338 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f6d0fac5-876a-4a14-a06f-1db0c0b7d3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:40.401397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 818/1650] 10.222.0.17 () {58 vars in 1059 bytes} [Fri Sep 9 13:14:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:40.408852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f80ba8-940b-46eb-ac27-4e7585886344 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:40.409171 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f80ba8-940b-46eb-ac27-4e7585886344 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:40.409451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f80ba8-940b-46eb-ac27-4e7585886344 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:40.411134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 833/1651] 10.222.0.17 () {54 vars in 912 bytes} [Fri Sep 9 13:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:14:40.416955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743c9c8-d1e0-4a08-beb1-3ac68a11ae9c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:40.417242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743c9c8-d1e0-4a08-beb1-3ac68a11ae9c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:40.417516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743c9c8-d1e0-4a08-beb1-3ac68a11ae9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:40.454625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0743c9c8-d1e0-4a08-beb1-3ac68a11ae9c None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:40.458178 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0743c9c8-d1e0-4a08-beb1-3ac68a11ae9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:40.484574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 819/1652] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:14:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:40.516421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f80ba8-940b-46eb-ac27-4e7585886344 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:40.542296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f80ba8-940b-46eb-ac27-4e7585886344 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:40.553436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3059af3-d931-4414-a084-d29def552126 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:40.554683 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3059af3-d931-4414-a084-d29def552126 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:40.555351 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3059af3-d931-4414-a084-d29def552126 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:40.555993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3059af3-d931-4414-a084-d29def552126 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:40.571875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3059af3-d931-4414-a084-d29def552126 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:40.573624 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3059af3-d931-4414-a084-d29def552126 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:40.619410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 834/1653] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:44.997610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25193ac4-0266-4423-897e-6b58f34fa283 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:44.998343 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25193ac4-0266-4423-897e-6b58f34fa283 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:44.998501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25193ac4-0266-4423-897e-6b58f34fa283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:45.043491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-25193ac4-0266-4423-897e-6b58f34fa283 None None] MFA Rules not processed for user `c31bb28e38864eff99fdd6938b1b89a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:45.045035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-25193ac4-0266-4423-897e-6b58f34fa283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:45.314571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 820/1654] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:45.336269 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3059af3-d931-4414-a084-d29def552126 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:45.352765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3059af3-d931-4414-a084-d29def552126 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:45.368761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f02b67-a726-48ee-84ec-d1f02e6f16b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:45.369389 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f02b67-a726-48ee-84ec-d1f02e6f16b2 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:45.369669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f02b67-a726-48ee-84ec-d1f02e6f16b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:45.369940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f02b67-a726-48ee-84ec-d1f02e6f16b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:45.387577 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f02b67-a726-48ee-84ec-d1f02e6f16b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:45.389372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f02b67-a726-48ee-84ec-d1f02e6f16b2 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:45.416340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 835/1655] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:47.082516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c09e2d-f1a1-4543-a159-3ebbfa934e98 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:47.083009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c09e2d-f1a1-4543-a159-3ebbfa934e98 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:47.083333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c09e2d-f1a1-4543-a159-3ebbfa934e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:47.145635 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-17c09e2d-f1a1-4543-a159-3ebbfa934e98 None None] MFA Rules not processed for user `efeb17c5a72b4e9294a63cf185737f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:47.148209 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-17c09e2d-f1a1-4543-a159-3ebbfa934e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:47.216680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:47.382393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 821/1656] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:47.401241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a366904-e88c-4f2a-bf43-7e720be516be None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:47.403854 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a366904-e88c-4f2a-bf43-7e720be516be None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:47.404630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a366904-e88c-4f2a-bf43-7e720be516be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:47.458185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6a366904-e88c-4f2a-bf43-7e720be516be None None] MFA Rules not processed for user `efeb17c5a72b4e9294a63cf185737f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:47.461301 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6a366904-e88c-4f2a-bf43-7e720be516be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:47.482981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 836/1657] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:47.498872 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c09e2d-f1a1-4543-a159-3ebbfa934e98 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:47.522017 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c09e2d-f1a1-4543-a159-3ebbfa934e98 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:47.530901 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a70dee-9ac5-4b2b-a380-db292231bbeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:47.532786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a70dee-9ac5-4b2b-a380-db292231bbeb service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:47.532786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a70dee-9ac5-4b2b-a380-db292231bbeb service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:47.532786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a70dee-9ac5-4b2b-a380-db292231bbeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:47.554365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a70dee-9ac5-4b2b-a380-db292231bbeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:47.558002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a70dee-9ac5-4b2b-a380-db292231bbeb service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:47.573259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 822/1658] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:14:48.899008 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b10ee3-22cc-4912-bbd7-3c46130db093 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:48.899961 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b10ee3-22cc-4912-bbd7-3c46130db093 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:48.900404 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b10ee3-22cc-4912-bbd7-3c46130db093 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:48.902188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 837/1659] 10.222.0.17 () {54 vars in 912 bytes} [Fri Sep 9 13:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:14:48.908197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc72b8-91ce-419d-b683-4a4cff120320 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:48.908633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc72b8-91ce-419d-b683-4a4cff120320 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:48.908764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc72b8-91ce-419d-b683-4a4cff120320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:48.913151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e049b4-da4a-433e-bbf2-5bb59cec9a53 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:48.914401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e049b4-da4a-433e-bbf2-5bb59cec9a53 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:48.915103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e049b4-da4a-433e-bbf2-5bb59cec9a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:48.960984 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-79bc72b8-91ce-419d-b683-4a4cff120320 None None] MFA Rules not processed for user `c31bb28e38864eff99fdd6938b1b89a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:48.962326 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-79bc72b8-91ce-419d-b683-4a4cff120320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:48.991367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 823/1660] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:49.025906 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-32e049b4-da4a-433e-bbf2-5bb59cec9a53 None None] MFA Rules not processed for user `e12deabe077b4095bfa7e2e8fac61d14`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:49.030476 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-32e049b4-da4a-433e-bbf2-5bb59cec9a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:49.048671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:49.166788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 838/1661] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:14:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:14:49.205727 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc72b8-91ce-419d-b683-4a4cff120320 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:49.219980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc72b8-91ce-419d-b683-4a4cff120320 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:49.227278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126da67-201c-4955-805f-165ce90b53b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:49.236512 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126da67-201c-4955-805f-165ce90b53b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:49.237262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126da67-201c-4955-805f-165ce90b53b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:49.237707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126da67-201c-4955-805f-165ce90b53b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:49.254369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126da67-201c-4955-805f-165ce90b53b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:49.255931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126da67-201c-4955-805f-165ce90b53b3 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:49.276547 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 824/1662] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:49.948683 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132226b-e43a-40e5-8932-e7b61e9715e7 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:49.949249 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132226b-e43a-40e5-8932-e7b61e9715e7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:49.950325 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132226b-e43a-40e5-8932-e7b61e9715e7 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:49.952135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 839/1663] 10.222.0.17 () {54 vars in 912 bytes} [Fri Sep 9 13:14:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:14:49.958305 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ced320-dcf4-42a2-82d7-7279ece349d8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:49.959360 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ced320-dcf4-42a2-82d7-7279ece349d8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:49.960915 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ced320-dcf4-42a2-82d7-7279ece349d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:50.008258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-60ced320-dcf4-42a2-82d7-7279ece349d8 None None] MFA Rules not processed for user `e12deabe077b4095bfa7e2e8fac61d14`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:50.009948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-60ced320-dcf4-42a2-82d7-7279ece349d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:50.030893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 825/1664] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:14:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:50.064596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132226b-e43a-40e5-8932-e7b61e9715e7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:50.105227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132226b-e43a-40e5-8932-e7b61e9715e7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:50.117876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a30839-d3b6-40fe-86e2-d926d4eddaf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:50.118905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a30839-d3b6-40fe-86e2-d926d4eddaf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:50.119206 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a30839-d3b6-40fe-86e2-d926d4eddaf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:50.119898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a30839-d3b6-40fe-86e2-d926d4eddaf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:50.129909 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a30839-d3b6-40fe-86e2-d926d4eddaf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:50.138969 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a30839-d3b6-40fe-86e2-d926d4eddaf3 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:50.160812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 840/1665] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:50.670899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ced320-dcf4-42a2-82d7-7279ece349d8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:50.888956 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ced320-dcf4-42a2-82d7-7279ece349d8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:50.904482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eac910-f8e1-4f90-9a69-9a2f42d4c472 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:50.905138 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eac910-f8e1-4f90-9a69-9a2f42d4c472 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:50.905414 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eac910-f8e1-4f90-9a69-9a2f42d4c472 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:50.905684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eac910-f8e1-4f90-9a69-9a2f42d4c472 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:50.911178 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e7eac910-f8e1-4f90-9a69-9a2f42d4c472 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:50.940366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eac910-f8e1-4f90-9a69-9a2f42d4c472 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:50.942179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eac910-f8e1-4f90-9a69-9a2f42d4c472 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:50.966206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 826/1666] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:52.821682 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67d24d-a93c-408f-bf1e-a7dcff155817 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:52.822157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67d24d-a93c-408f-bf1e-a7dcff155817 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:52.822229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67d24d-a93c-408f-bf1e-a7dcff155817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:52.921564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ba67d24d-a93c-408f-bf1e-a7dcff155817 None None] MFA Rules not processed for user `21700a3c2ff24fc6a3bb5db8bf283f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:52.925646 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ba67d24d-a93c-408f-bf1e-a7dcff155817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:53.100282 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be5c04-133b-41c3-94a7-68a548bf7673 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:53.101198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be5c04-133b-41c3-94a7-68a548bf7673 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:53.101699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be5c04-133b-41c3-94a7-68a548bf7673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:53.103275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 841/1667] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:53.153700 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-22be5c04-133b-41c3-94a7-68a548bf7673 None None] MFA Rules not processed for user `21700a3c2ff24fc6a3bb5db8bf283f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:53.155150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-22be5c04-133b-41c3-94a7-68a548bf7673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:53.183891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 827/1668] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:53.229271 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67d24d-a93c-408f-bf1e-a7dcff155817 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:53.294721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67d24d-a93c-408f-bf1e-a7dcff155817 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:53.323548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f3c34-7f67-4f3c-81e1-21d3c1d0da6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:53.323769 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f3c34-7f67-4f3c-81e1-21d3c1d0da6f service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:53.329751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f3c34-7f67-4f3c-81e1-21d3c1d0da6f service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:53.332116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f3c34-7f67-4f3c-81e1-21d3c1d0da6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:53.397589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be5c04-133b-41c3-94a7-68a548bf7673 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:53.400472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f3c34-7f67-4f3c-81e1-21d3c1d0da6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:53.408594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f3c34-7f67-4f3c-81e1-21d3c1d0da6f service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:53.460871 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 842/1669] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:53 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:14:53.645135 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be5c04-133b-41c3-94a7-68a548bf7673 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:53.659417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905ff3e-35eb-4ab1-b16e-6eb0437bf530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:53.663526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905ff3e-35eb-4ab1-b16e-6eb0437bf530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:53.664494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905ff3e-35eb-4ab1-b16e-6eb0437bf530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:53.665286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905ff3e-35eb-4ab1-b16e-6eb0437bf530 admin admin] PATH_INFO: `/v3/users/021f677dcf064541835e9a68efe5b145` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:53.670891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905ff3e-35eb-4ab1-b16e-6eb0437bf530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=021f677dcf064541835e9a68efe5b145)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:53.673582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-8905ff3e-35eb-4ab1-b16e-6eb0437bf530 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:14:53.673582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Sep 09 13:14:53.673582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: Sep 09 13:14:53.677071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905ff3e-35eb-4ab1-b16e-6eb0437bf530 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:53.700260 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:53.842167 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8905ff3e-35eb-4ab1-b16e-6eb0437bf530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '021f677dcf064541835e9a68efe5b145'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:53.860629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8905ff3e-35eb-4ab1-b16e-6eb0437bf530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '021f677dcf064541835e9a68efe5b145'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:53.897903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650e913-cb0a-4ed9-a854-5839c99389e5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:53.899143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650e913-cb0a-4ed9-a854-5839c99389e5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:53.899637 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650e913-cb0a-4ed9-a854-5839c99389e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:53.900791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8905ff3e-35eb-4ab1-b16e-6eb0437bf530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '021f677dcf064541835e9a68efe5b145'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:53.901715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8905ff3e-35eb-4ab1-b16e-6eb0437bf530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '021f677dcf064541835e9a68efe5b145'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:53.932147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8905ff3e-35eb-4ab1-b16e-6eb0437bf530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '021f677dcf064541835e9a68efe5b145'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:53.952939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 828/1670] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:14:53 2022] DELETE /identity/v3/users/021f677dcf064541835e9a68efe5b145 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:53.954626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f32c2-7a9a-4f26-9af2-3ce4f6fc616b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:53.955076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f32c2-7a9a-4f26-9af2-3ce4f6fc616b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:53.955500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f32c2-7a9a-4f26-9af2-3ce4f6fc616b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:54.002003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6650e913-cb0a-4ed9-a854-5839c99389e5 None None] MFA Rules not processed for user `dcb88983a8e1439588644b71170d7362`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:54.003735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6650e913-cb0a-4ed9-a854-5839c99389e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:54.036878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-292f32c2-7a9a-4f26-9af2-3ce4f6fc616b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:54.038147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-292f32c2-7a9a-4f26-9af2-3ce4f6fc616b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:54.147623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 829/1671] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:54.154906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f32c2-7a9a-4f26-9af2-3ce4f6fc616b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:54.157194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-292f32c2-7a9a-4f26-9af2-3ce4f6fc616b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:54.246163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 843/1672] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:14:54.266931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650e913-cb0a-4ed9-a854-5839c99389e5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:54.268603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6650e913-cb0a-4ed9-a854-5839c99389e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:54.291486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:54.360509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f32c2-7a9a-4f26-9af2-3ce4f6fc616b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:54.382461 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbe0ec-44ef-4bbc-a34d-70dae04187ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:54.383032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbe0ec-44ef-4bbc-a34d-70dae04187ca service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:54.383346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbe0ec-44ef-4bbc-a34d-70dae04187ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:54.383619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbe0ec-44ef-4bbc-a34d-70dae04187ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:54.424324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbe0ec-44ef-4bbc-a34d-70dae04187ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:54.427225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbe0ec-44ef-4bbc-a34d-70dae04187ca service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:54.491608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 830/1673] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:54.530489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650e913-cb0a-4ed9-a854-5839c99389e5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:54.575760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff9bea-1c28-40c2-b186-e0ef100e8223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:54.576783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff9bea-1c28-40c2-b186-e0ef100e8223 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:54.579283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff9bea-1c28-40c2-b186-e0ef100e8223 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:54.579774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff9bea-1c28-40c2-b186-e0ef100e8223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:54.665971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff9bea-1c28-40c2-b186-e0ef100e8223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:54.673205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff9bea-1c28-40c2-b186-e0ef100e8223 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:54.742229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 844/1674] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:14:55.044087 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbe0ec-44ef-4bbc-a34d-70dae04187ca service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:55.045828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-17cbe0ec-44ef-4bbc-a34d-70dae04187ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:55.081655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbe0ec-44ef-4bbc-a34d-70dae04187ca service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:55.095926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faafb5d-47af-48d2-b3b6-2c7473c4f282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:55.101103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faafb5d-47af-48d2-b3b6-2c7473c4f282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:55.101429 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faafb5d-47af-48d2-b3b6-2c7473c4f282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:55.101708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faafb5d-47af-48d2-b3b6-2c7473c4f282 admin admin] PATH_INFO: `/v3/users/2a98941032454d0f969f3cde046bbaf0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:55.109884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faafb5d-47af-48d2-b3b6-2c7473c4f282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a98941032454d0f969f3cde046bbaf0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:55.122970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faafb5d-47af-48d2-b3b6-2c7473c4f282 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:55.223747 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:55.357036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0faafb5d-47af-48d2-b3b6-2c7473c4f282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a98941032454d0f969f3cde046bbaf0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:55.381901 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5287f3-fb96-4ec0-9d70-ae928e088e3c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:55.382563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5287f3-fb96-4ec0-9d70-ae928e088e3c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:55.382875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5287f3-fb96-4ec0-9d70-ae928e088e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:55.387941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0faafb5d-47af-48d2-b3b6-2c7473c4f282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a98941032454d0f969f3cde046bbaf0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:55.422238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0faafb5d-47af-48d2-b3b6-2c7473c4f282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a98941032454d0f969f3cde046bbaf0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:55.424541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0faafb5d-47af-48d2-b3b6-2c7473c4f282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a98941032454d0f969f3cde046bbaf0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:55.455580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0faafb5d-47af-48d2-b3b6-2c7473c4f282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a98941032454d0f969f3cde046bbaf0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:55.474085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 831/1675] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:14:55 2022] DELETE /identity/v3/users/2a98941032454d0f969f3cde046bbaf0 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:55.476141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbd5ec-87a7-4851-a1f3-7701321e94b6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:55.476679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbd5ec-87a7-4851-a1f3-7701321e94b6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:55.477206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbd5ec-87a7-4851-a1f3-7701321e94b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:55.570363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6ecbd5ec-87a7-4851-a1f3-7701321e94b6 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:55.572911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6ecbd5ec-87a7-4851-a1f3-7701321e94b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:55.664859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2f5287f3-fb96-4ec0-9d70-ae928e088e3c None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:55.669012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2f5287f3-fb96-4ec0-9d70-ae928e088e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:55.799981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 832/1676] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:55.833856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbd5ec-87a7-4851-a1f3-7701321e94b6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:55.846767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6ecbd5ec-87a7-4851-a1f3-7701321e94b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:55.930919 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 845/1677] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:14:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 551 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:14:55.932612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b735b9e-482d-48a8-827e-6e5500b0413b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:55.932968 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b735b9e-482d-48a8-827e-6e5500b0413b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:55.933735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b735b9e-482d-48a8-827e-6e5500b0413b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:56.004766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8b735b9e-482d-48a8-827e-6e5500b0413b None None] MFA Rules not processed for user `21700a3c2ff24fc6a3bb5db8bf283f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:56.013729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8b735b9e-482d-48a8-827e-6e5500b0413b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:56.040454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbd5ec-87a7-4851-a1f3-7701321e94b6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:56.056858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4eade-a289-4054-a63d-2c1429a9ead8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:56.057500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4eade-a289-4054-a63d-2c1429a9ead8 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:56.060894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4eade-a289-4054-a63d-2c1429a9ead8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:56.061239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4eade-a289-4054-a63d-2c1429a9ead8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:56.111457 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4eade-a289-4054-a63d-2c1429a9ead8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:56.112917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4eade-a289-4054-a63d-2c1429a9ead8 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:56.144279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 833/1678] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:14:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:14:56.213223 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 846/1679] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:56.321110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4eade-a289-4054-a63d-2c1429a9ead8 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:56.322623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a6b4eade-a289-4054-a63d-2c1429a9ead8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:56.495208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:56.646082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4eade-a289-4054-a63d-2c1429a9ead8 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:56.665479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c95e5-3577-4b40-9709-793ef7e2ba9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:56.666346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c95e5-3577-4b40-9709-793ef7e2ba9f service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:56.666541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c95e5-3577-4b40-9709-793ef7e2ba9f service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:56.666805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c95e5-3577-4b40-9709-793ef7e2ba9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:56.686366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c95e5-3577-4b40-9709-793ef7e2ba9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:56.689976 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c95e5-3577-4b40-9709-793ef7e2ba9f service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:56.713581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 834/1680] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:14:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:14:58.948708 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b735b9e-482d-48a8-827e-6e5500b0413b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:58.951082 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8b735b9e-482d-48a8-827e-6e5500b0413b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:58.997389 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b735b9e-482d-48a8-827e-6e5500b0413b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:59.012671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d81f6-0d81-403b-9baf-465526256ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:59.013410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d81f6-0d81-403b-9baf-465526256ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:59.013731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d81f6-0d81-403b-9baf-465526256ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:59.016393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d81f6-0d81-403b-9baf-465526256ebf admin admin] PATH_INFO: `/v3/projects/995f9843e1994072b79c1f0b38c96140` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:59.021266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d81f6-0d81-403b-9baf-465526256ebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=995f9843e1994072b79c1f0b38c96140)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:59.022407 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-4d5d81f6-0d81-403b-9baf-465526256ebf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 13:14:59.022407 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Sep 09 13:14:59.022407 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: Sep 09 13:14:59.024576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d81f6-0d81-403b-9baf-465526256ebf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:59.089858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:14:59.309708 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4d5d81f6-0d81-403b-9baf-465526256ebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '995f9843e1994072b79c1f0b38c96140', 'request_id': 'req-4d5d81f6-0d81-403b-9baf-465526256ebf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:59.327575 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4d5d81f6-0d81-403b-9baf-465526256ebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '995f9843e1994072b79c1f0b38c96140', 'request_id': 'req-4d5d81f6-0d81-403b-9baf-465526256ebf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:59.338089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4d5d81f6-0d81-403b-9baf-465526256ebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '995f9843e1994072b79c1f0b38c96140', 'request_id': 'req-4d5d81f6-0d81-403b-9baf-465526256ebf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:59.396180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4d5d81f6-0d81-403b-9baf-465526256ebf admin admin] The token cache is being invalidate because project 995f9843e1994072b79c1f0b38c96140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:14:59.396854 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4d5d81f6-0d81-403b-9baf-465526256ebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:59.397632 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4d5d81f6-0d81-403b-9baf-465526256ebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:14:59.400286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 847/1681] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:14:58 2022] DELETE /identity/v3/projects/995f9843e1994072b79c1f0b38c96140 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:14:59.405656 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde3338-2491-4d51-8fed-447f998750da None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:59.406653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde3338-2491-4d51-8fed-447f998750da None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:59.406933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde3338-2491-4d51-8fed-447f998750da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:59.466362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ecde3338-2491-4d51-8fed-447f998750da None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:59.472060 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ecde3338-2491-4d51-8fed-447f998750da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:59.691431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 835/1682] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:59.700908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5a1a1-8720-4ebf-9caa-150cf8eeb44d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:59.702243 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5a1a1-8720-4ebf-9caa-150cf8eeb44d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:59.702676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5a1a1-8720-4ebf-9caa-150cf8eeb44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:59.761539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-12e5a1a1-8720-4ebf-9caa-150cf8eeb44d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:14:59.768314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-12e5a1a1-8720-4ebf-9caa-150cf8eeb44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:14:59.816905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 848/1683] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:14:59.821025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde3338-2491-4d51-8fed-447f998750da None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:59.850418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde3338-2491-4d51-8fed-447f998750da None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:59.883679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a997c6-52e3-40f4-bc63-af3568f3a5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:14:59.886026 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a997c6-52e3-40f4-bc63-af3568f3a5b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:14:59.888044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a997c6-52e3-40f4-bc63-af3568f3a5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:14:59.888197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a997c6-52e3-40f4-bc63-af3568f3a5b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:14:59.888935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a997c6-52e3-40f4-bc63-af3568f3a5b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:14:59.889393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a997c6-52e3-40f4-bc63-af3568f3a5b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:14:59.895782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a997c6-52e3-40f4-bc63-af3568f3a5b5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:14:59.962326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5a1a1-8720-4ebf-9caa-150cf8eeb44d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:14:59.976567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 836/1684] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:14:59.980089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5a1a1-8720-4ebf-9caa-150cf8eeb44d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:14:59.996118 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0834e-4f8d-4735-8191-f258b7f92db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:00.000052 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0834e-4f8d-4735-8191-f258b7f92db7 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:00.000668 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0834e-4f8d-4735-8191-f258b7f92db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:00.000857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0834e-4f8d-4735-8191-f258b7f92db7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:00.001868 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0834e-4f8d-4735-8191-f258b7f92db7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:00.005660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0834e-4f8d-4735-8191-f258b7f92db7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:00.074127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a997c6-52e3-40f4-bc63-af3568f3a5b5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:00.074276 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 849/1685] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:14:59 2022] POST /identity/v3/projects => generated 403 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:00.104826 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a997c6-52e3-40f4-bc63-af3568f3a5b5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:00.319807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb765fa-31d9-48dd-aac5-4fb82bfbdac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:00.324243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb765fa-31d9-48dd-aac5-4fb82bfbdac7 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:00.327132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb765fa-31d9-48dd-aac5-4fb82bfbdac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:00.328272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb765fa-31d9-48dd-aac5-4fb82bfbdac7 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:00.330990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb765fa-31d9-48dd-aac5-4fb82bfbdac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:00.334531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb765fa-31d9-48dd-aac5-4fb82bfbdac7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:00.416103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0834e-4f8d-4735-8191-f258b7f92db7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:00.417880 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 837/1686] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:00 2022] POST /identity/v3/users => generated 302 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:00.433801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0834e-4f8d-4735-8191-f258b7f92db7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:00.454326 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb765fa-31d9-48dd-aac5-4fb82bfbdac7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:00.456907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bdb765fa-31d9-48dd-aac5-4fb82bfbdac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:00.462637 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cae8e1-0871-40c9-93e3-9a49e2fa5aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:00.463603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cae8e1-0871-40c9-93e3-9a49e2fa5aad admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:00.463948 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cae8e1-0871-40c9-93e3-9a49e2fa5aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:00.464395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cae8e1-0871-40c9-93e3-9a49e2fa5aad admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:00.465113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cae8e1-0871-40c9-93e3-9a49e2fa5aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:00.471367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cae8e1-0871-40c9-93e3-9a49e2fa5aad admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:00.500886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb765fa-31d9-48dd-aac5-4fb82bfbdac7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:00.516329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 850/1687] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:00.517388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cae8e1-0871-40c9-93e3-9a49e2fa5aad admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:00.532388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:00.532995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:00.534284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:00.536354 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] PATH_INFO: `/v3/projects/f6e2f1f36448419ea1d12bbc8e4d1100` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:00.542674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6e2f1f36448419ea1d12bbc8e4d1100)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:00.548644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cae8e1-0871-40c9-93e3-9a49e2fa5aad admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:00.551293 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:00.562167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90e1b3-c239-457b-8c69-b653b0be1d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:00.562934 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90e1b3-c239-457b-8c69-b653b0be1d06 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:00.563369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90e1b3-c239-457b-8c69-b653b0be1d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:00.563734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90e1b3-c239-457b-8c69-b653b0be1d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:00.564480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90e1b3-c239-457b-8c69-b653b0be1d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:00.567262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90e1b3-c239-457b-8c69-b653b0be1d06 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:00.594970 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 851/1688] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:00.598047 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87822595-403e-4d77-8426-39cb9b9b3252 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:00.598849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87822595-403e-4d77-8426-39cb9b9b3252 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:00.599274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87822595-403e-4d77-8426-39cb9b9b3252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:00.651480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-87822595-403e-4d77-8426-39cb9b9b3252 None None] MFA Rules not processed for user `e12deabe077b4095bfa7e2e8fac61d14`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:00.652956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-87822595-403e-4d77-8426-39cb9b9b3252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:00.656982 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:00.767578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6e2f1f36448419ea1d12bbc8e4d1100', 'request_id': 'req-e4fc5167-9188-4fea-988e-941e982a2a4d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:00.803388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6e2f1f36448419ea1d12bbc8e4d1100', 'request_id': 'req-e4fc5167-9188-4fea-988e-941e982a2a4d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:00.821664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6e2f1f36448419ea1d12bbc8e4d1100', 'request_id': 'req-e4fc5167-9188-4fea-988e-941e982a2a4d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:00.854920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 852/1689] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Sep 9 13:15:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:00.858831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87822595-403e-4d77-8426-39cb9b9b3252 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:00.885572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] The token cache is being invalidate because project f6e2f1f36448419ea1d12bbc8e4d1100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:00.896698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:00.899762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:00.900051 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87822595-403e-4d77-8426-39cb9b9b3252 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:00.905290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 838/1690] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:00 2022] DELETE /identity/v3/projects/f6e2f1f36448419ea1d12bbc8e4d1100 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:00.907071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:00.910524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:01.073425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4811bfb-039f-4ae2-baab-a497af47723b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:01.074175 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4811bfb-039f-4ae2-baab-a497af47723b admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:01.074396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4811bfb-039f-4ae2-baab-a497af47723b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:01.077027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4811bfb-039f-4ae2-baab-a497af47723b admin admin] PATH_INFO: `/v3/projects/450f20a90a69450bab035066515b191e/users/e385230391674db6ab1c0589534e19a2/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:01.079975 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:01.119682 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4811bfb-039f-4ae2-baab-a497af47723b admin admin] RBAC: Authorizing `identity:create_grant(project_id=450f20a90a69450bab035066515b191e, user_id=e385230391674db6ab1c0589534e19a2, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:01.123130 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4811bfb-039f-4ae2-baab-a497af47723b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:01.177195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 853/1691] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:00 2022] PUT /identity/v3/projects/450f20a90a69450bab035066515b191e/users/e385230391674db6ab1c0589534e19a2/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:01.178707 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4811bfb-039f-4ae2-baab-a497af47723b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:01.192492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc5167-9188-4fea-988e-941e982a2a4d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:01.196005 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c4811bfb-039f-4ae2-baab-a497af47723b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:01.409267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc6c8d-a5e2-4688-91e4-a406b279c68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:01.411282 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc6c8d-a5e2-4688-91e4-a406b279c68e service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:01.413182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc6c8d-a5e2-4688-91e4-a406b279c68e service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:01.414014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc6c8d-a5e2-4688-91e4-a406b279c68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:01.420449 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-91fc6c8d-a5e2-4688-91e4-a406b279c68e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:01.441468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4811bfb-039f-4ae2-baab-a497af47723b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:01.457410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc6c8d-a5e2-4688-91e4-a406b279c68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:01.459719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:01.461108 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc6c8d-a5e2-4688-91e4-a406b279c68e service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:01.461859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:01.462276 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:01.463034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] PATH_INFO: `/v3/users/3a0d5f56cd544425bd0f96e5063d206c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:01.466386 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a0d5f56cd544425bd0f96e5063d206c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:01.469945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:01.497697 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 839/1692] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:01.502480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9e27e-f669-437c-b28b-179d8ad79908 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:01.502769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9e27e-f669-437c-b28b-179d8ad79908 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:01.503078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9e27e-f669-437c-b28b-179d8ad79908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:01.566375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-69c9e27e-f669-437c-b28b-179d8ad79908 None None] MFA Rules not processed for user `e385230391674db6ab1c0589534e19a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:01.568504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-69c9e27e-f669-437c-b28b-179d8ad79908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:01.623529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a0d5f56cd544425bd0f96e5063d206c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:01.664427 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a0d5f56cd544425bd0f96e5063d206c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:01.666236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a0d5f56cd544425bd0f96e5063d206c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:01.706879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a0d5f56cd544425bd0f96e5063d206c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:01.727653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a0d5f56cd544425bd0f96e5063d206c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:01.756978 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 854/1693] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:01 2022] DELETE /identity/v3/users/3a0d5f56cd544425bd0f96e5063d206c => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:01.830687 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:01.846635 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 840/1694] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:01 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:01.852066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:01.854020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:02.011760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a4f00-3e57-4a6d-b2ae-f186d386822c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:02.035678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededcf11-22fd-41ba-8fba-53101711ebbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:02.037905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededcf11-22fd-41ba-8fba-53101711ebbf admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:02.038680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededcf11-22fd-41ba-8fba-53101711ebbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:02.039252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededcf11-22fd-41ba-8fba-53101711ebbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:02.040014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededcf11-22fd-41ba-8fba-53101711ebbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:02.044227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededcf11-22fd-41ba-8fba-53101711ebbf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:02.053642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:02.142328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 855/1695] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:01 2022] POST /identity/v3/projects => generated 405 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:02.143921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9e27e-f669-437c-b28b-179d8ad79908 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:02.174961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9e27e-f669-437c-b28b-179d8ad79908 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:02.513339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bae0f8-9cc9-45ac-bb1a-51fce24f9956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:02.516328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bae0f8-9cc9-45ac-bb1a-51fce24f9956 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:02.516910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bae0f8-9cc9-45ac-bb1a-51fce24f9956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:02.519571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bae0f8-9cc9-45ac-bb1a-51fce24f9956 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:02.520300 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bae0f8-9cc9-45ac-bb1a-51fce24f9956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:02.535647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bae0f8-9cc9-45ac-bb1a-51fce24f9956 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:02.634055 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 841/1696] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:02 2022] POST /identity/v3/users => generated 309 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:02.638845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededcf11-22fd-41ba-8fba-53101711ebbf admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:02.665568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededcf11-22fd-41ba-8fba-53101711ebbf admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:02.678185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8257ef-54d9-43cc-a342-a682f82ffd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:02.679169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8257ef-54d9-43cc-a342-a682f82ffd1b admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:02.679649 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8257ef-54d9-43cc-a342-a682f82ffd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:02.680474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8257ef-54d9-43cc-a342-a682f82ffd1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:02.681379 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8257ef-54d9-43cc-a342-a682f82ffd1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:02.683249 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8257ef-54d9-43cc-a342-a682f82ffd1b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:02.711738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 856/1697] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:02.716145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bae0f8-9cc9-45ac-bb1a-51fce24f9956 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:02.727477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bae0f8-9cc9-45ac-bb1a-51fce24f9956 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:02.734629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433735c7-9261-4be3-8316-3b50b0b10f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:02.735770 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433735c7-9261-4be3-8316-3b50b0b10f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:02.736117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433735c7-9261-4be3-8316-3b50b0b10f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:02.736746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433735c7-9261-4be3-8316-3b50b0b10f82 admin admin] PATH_INFO: `/v3/projects/591239d3fe764bfcb6ef60ac30c25eed/users/1b59460ac6244d19a597b681ce924c81/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:02.772198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433735c7-9261-4be3-8316-3b50b0b10f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=591239d3fe764bfcb6ef60ac30c25eed, user_id=1b59460ac6244d19a597b681ce924c81, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:02.775611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433735c7-9261-4be3-8316-3b50b0b10f82 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:02.819131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 842/1698] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:02 2022] PUT /identity/v3/projects/591239d3fe764bfcb6ef60ac30c25eed/users/1b59460ac6244d19a597b681ce924c81/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:02.827350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8257ef-54d9-43cc-a342-a682f82ffd1b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:02.842126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8257ef-54d9-43cc-a342-a682f82ffd1b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:03.082104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df15168-e684-4ee4-93dc-15e56b2d8de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:03.082768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df15168-e684-4ee4-93dc-15e56b2d8de3 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:03.083438 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df15168-e684-4ee4-93dc-15e56b2d8de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:03.083438 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df15168-e684-4ee4-93dc-15e56b2d8de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:03.090819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df15168-e684-4ee4-93dc-15e56b2d8de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:03.092405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df15168-e684-4ee4-93dc-15e56b2d8de3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:03.114162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 857/1699] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:03.117549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433735c7-9261-4be3-8316-3b50b0b10f82 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:03.154725 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433735c7-9261-4be3-8316-3b50b0b10f82 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:03.169219 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880cbf7-96b7-4842-8beb-682bcf3621a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:03.170139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880cbf7-96b7-4842-8beb-682bcf3621a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:03.171880 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880cbf7-96b7-4842-8beb-682bcf3621a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:03.172506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880cbf7-96b7-4842-8beb-682bcf3621a6 admin admin] PATH_INFO: `/v3/projects/591239d3fe764bfcb6ef60ac30c25eed/users/1b59460ac6244d19a597b681ce924c81/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:03.176383 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880cbf7-96b7-4842-8beb-682bcf3621a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=591239d3fe764bfcb6ef60ac30c25eed, user_id=1b59460ac6244d19a597b681ce924c81, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:03.179581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880cbf7-96b7-4842-8beb-682bcf3621a6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:03.226933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 843/1700] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:03 2022] PUT /identity/v3/projects/591239d3fe764bfcb6ef60ac30c25eed/users/1b59460ac6244d19a597b681ce924c81/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:03.233478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b06dc-e15c-4362-ba89-b38fac5801f3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:03.234529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b06dc-e15c-4362-ba89-b38fac5801f3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:03.234968 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b06dc-e15c-4362-ba89-b38fac5801f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:03.302767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-718b06dc-e15c-4362-ba89-b38fac5801f3 None None] MFA Rules not processed for user `1b59460ac6244d19a597b681ce924c81`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:03.305126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-718b06dc-e15c-4362-ba89-b38fac5801f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:03.520261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 858/1701] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:03 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:03.532846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880cbf7-96b7-4842-8beb-682bcf3621a6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:03.553109 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2880cbf7-96b7-4842-8beb-682bcf3621a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:03.832333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880cbf7-96b7-4842-8beb-682bcf3621a6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:03.844116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5348b-69cf-48b5-94e4-482d1a888dc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:03.846385 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5348b-69cf-48b5-94e4-482d1a888dc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:03.846786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5348b-69cf-48b5-94e4-482d1a888dc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:03.849861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5348b-69cf-48b5-94e4-482d1a888dc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:03.881589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a5348b-69cf-48b5-94e4-482d1a888dc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:03.894137 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a5348b-69cf-48b5-94e4-482d1a888dc8 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:03.927938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 844/1702] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:03 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:04.719277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b06dc-e15c-4362-ba89-b38fac5801f3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:04.752686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5348b-69cf-48b5-94e4-482d1a888dc8 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:04.754228 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-09a5348b-69cf-48b5-94e4-482d1a888dc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:05.055163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:05.069282 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:05.074298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b06dc-e15c-4362-ba89-b38fac5801f3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:05.096893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5348b-69cf-48b5-94e4-482d1a888dc8 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:05.098742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1593d63-5e12-4de8-bd3d-4ca0cdd01b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:05.100464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1593d63-5e12-4de8-bd3d-4ca0cdd01b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:05.100808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1593d63-5e12-4de8-bd3d-4ca0cdd01b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:05.101526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1593d63-5e12-4de8-bd3d-4ca0cdd01b95 admin admin] PATH_INFO: `/v3/users/e385230391674db6ab1c0589534e19a2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:05.107021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1593d63-5e12-4de8-bd3d-4ca0cdd01b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e385230391674db6ab1c0589534e19a2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:05.110929 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1593d63-5e12-4de8-bd3d-4ca0cdd01b95 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:05.112090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8549b9-d508-40eb-a102-060250d818f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:05.112924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8549b9-d508-40eb-a102-060250d818f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:05.115430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8549b9-d508-40eb-a102-060250d818f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:05.115940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8549b9-d508-40eb-a102-060250d818f5 admin admin] PATH_INFO: `/v3/projects/ff462ec87a0541d8997d9ea9fd44aab3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:05.118750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8549b9-d508-40eb-a102-060250d818f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff462ec87a0541d8997d9ea9fd44aab3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:05.125361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8549b9-d508-40eb-a102-060250d818f5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:05.254472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c1593d63-5e12-4de8-bd3d-4ca0cdd01b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e385230391674db6ab1c0589534e19a2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:05.274453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c1593d63-5e12-4de8-bd3d-4ca0cdd01b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e385230391674db6ab1c0589534e19a2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:05.277013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c1593d63-5e12-4de8-bd3d-4ca0cdd01b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e385230391674db6ab1c0589534e19a2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:05.284346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2e8549b9-d508-40eb-a102-060250d818f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff462ec87a0541d8997d9ea9fd44aab3', 'request_id': 'req-2e8549b9-d508-40eb-a102-060250d818f5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:05.319643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2e8549b9-d508-40eb-a102-060250d818f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff462ec87a0541d8997d9ea9fd44aab3', 'request_id': 'req-2e8549b9-d508-40eb-a102-060250d818f5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:05.344963 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c1593d63-5e12-4de8-bd3d-4ca0cdd01b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e385230391674db6ab1c0589534e19a2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:05.349650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2e8549b9-d508-40eb-a102-060250d818f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff462ec87a0541d8997d9ea9fd44aab3', 'request_id': 'req-2e8549b9-d508-40eb-a102-060250d818f5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:05.377448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c1593d63-5e12-4de8-bd3d-4ca0cdd01b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e385230391674db6ab1c0589534e19a2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:05.396543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2e8549b9-d508-40eb-a102-060250d818f5 admin admin] The token cache is being invalidate because project ff462ec87a0541d8997d9ea9fd44aab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:05.405186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2e8549b9-d508-40eb-a102-060250d818f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:05.421156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2e8549b9-d508-40eb-a102-060250d818f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:05.432106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 845/1703] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:04 2022] DELETE /identity/v3/projects/ff462ec87a0541d8997d9ea9fd44aab3 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:05.433982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 859/1704] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:04 2022] DELETE /identity/v3/users/e385230391674db6ab1c0589534e19a2 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:05.435803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b682dfe-cf41-404b-9e0d-340e8efbb1cc None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:05.436097 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b682dfe-cf41-404b-9e0d-340e8efbb1cc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:05.436373 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b682dfe-cf41-404b-9e0d-340e8efbb1cc None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:05.438500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba7b2b-4cd6-4d6e-8aa6-89ff3f0fd786 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:05.438791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba7b2b-4cd6-4d6e-8aa6-89ff3f0fd786 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:05.449034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba7b2b-4cd6-4d6e-8aa6-89ff3f0fd786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:05.450966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 846/1705] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:15:05 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:15:05.458408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75221d92-9df6-404a-9606-68eda4f6b35c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:05.462340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75221d92-9df6-404a-9606-68eda4f6b35c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:05.462680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75221d92-9df6-404a-9606-68eda4f6b35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:05.539515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-33ba7b2b-4cd6-4d6e-8aa6-89ff3f0fd786 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:05.544971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-33ba7b2b-4cd6-4d6e-8aa6-89ff3f0fd786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:05.555980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-75221d92-9df6-404a-9606-68eda4f6b35c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:05.557683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-75221d92-9df6-404a-9606-68eda4f6b35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:05.777269 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 860/1706] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:05.786949 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028db98f-21ae-4404-96e0-a29d0f016f35 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:05.787294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028db98f-21ae-4404-96e0-a29d0f016f35 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:05.787628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028db98f-21ae-4404-96e0-a29d0f016f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:05.836409 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-028db98f-21ae-4404-96e0-a29d0f016f35 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:05.837692 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-028db98f-21ae-4404-96e0-a29d0f016f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:05.876369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 861/1707] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:05.880498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028db98f-21ae-4404-96e0-a29d0f016f35 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:05.899670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028db98f-21ae-4404-96e0-a29d0f016f35 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:05.905284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 847/1708] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:05.920377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20ab96-2691-472a-8a42-f19d15072d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:05.921375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20ab96-2691-472a-8a42-f19d15072d12 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:05.921905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20ab96-2691-472a-8a42-f19d15072d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:05.922336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20ab96-2691-472a-8a42-f19d15072d12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:05.923996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20ab96-2691-472a-8a42-f19d15072d12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:15:05.924615 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20ab96-2691-472a-8a42-f19d15072d12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:05.929144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20ab96-2691-472a-8a42-f19d15072d12 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:05.984671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 862/1709] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:15:05.986089 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75221d92-9df6-404a-9606-68eda4f6b35c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:05.995993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c5d3c-929d-42d1-960d-c3d111e7434d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:05.996291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c5d3c-929d-42d1-960d-c3d111e7434d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:05.996470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c5d3c-929d-42d1-960d-c3d111e7434d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:06.016321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75221d92-9df6-404a-9606-68eda4f6b35c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:06.038241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e299a6f-a1f0-49e8-a9bf-7dbb4bb2e290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:06.038905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e299a6f-a1f0-49e8-a9bf-7dbb4bb2e290 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:06.039374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e299a6f-a1f0-49e8-a9bf-7dbb4bb2e290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:06.039643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e299a6f-a1f0-49e8-a9bf-7dbb4bb2e290 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:06.040392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e299a6f-a1f0-49e8-a9bf-7dbb4bb2e290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:06.042357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e299a6f-a1f0-49e8-a9bf-7dbb4bb2e290 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:06.072595 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-783c5d3c-929d-42d1-960d-c3d111e7434d None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:06.075726 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-783c5d3c-929d-42d1-960d-c3d111e7434d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:06.165980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 848/1710] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:05 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:06.182670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e299a6f-a1f0-49e8-a9bf-7dbb4bb2e290 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:06.296130 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:06.374937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 863/1711] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:15:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:06.375904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c5d3c-929d-42d1-960d-c3d111e7434d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:06.387957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-783c5d3c-929d-42d1-960d-c3d111e7434d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:06.480729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e299a6f-a1f0-49e8-a9bf-7dbb4bb2e290 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:06.485435 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c5d3c-929d-42d1-960d-c3d111e7434d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:06.501420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70dc59-4eb6-4ec2-baa0-1d0a7a17632f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:06.501988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70dc59-4eb6-4ec2-baa0-1d0a7a17632f admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:06.502269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70dc59-4eb6-4ec2-baa0-1d0a7a17632f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:06.502636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70dc59-4eb6-4ec2-baa0-1d0a7a17632f admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:06.509630 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70dc59-4eb6-4ec2-baa0-1d0a7a17632f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:06.514580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70dc59-4eb6-4ec2-baa0-1d0a7a17632f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:06.571098 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:06.648730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 849/1712] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:06 2022] POST /identity/v3/users => generated 289 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:06.649960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70dc59-4eb6-4ec2-baa0-1d0a7a17632f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:06.687899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0841e1c-6ecf-46b1-8fb7-a89d7e9722df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:06.689000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0841e1c-6ecf-46b1-8fb7-a89d7e9722df service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:06.690840 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0841e1c-6ecf-46b1-8fb7-a89d7e9722df service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:06.692305 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0841e1c-6ecf-46b1-8fb7-a89d7e9722df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:06.695951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70dc59-4eb6-4ec2-baa0-1d0a7a17632f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:06.727461 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a3b1a-ec78-460b-8b34-90683e8bc06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:06.731861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a3b1a-ec78-460b-8b34-90683e8bc06a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:06.732319 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a3b1a-ec78-460b-8b34-90683e8bc06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:06.733755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a3b1a-ec78-460b-8b34-90683e8bc06a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:06.734583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a3b1a-ec78-460b-8b34-90683e8bc06a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:06.740788 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a3b1a-ec78-460b-8b34-90683e8bc06a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:06.742710 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0841e1c-6ecf-46b1-8fb7-a89d7e9722df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:06.744479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0841e1c-6ecf-46b1-8fb7-a89d7e9722df service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:06.788643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 850/1713] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:06.789860 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a3b1a-ec78-460b-8b34-90683e8bc06a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:06.796385 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 864/1714] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:06.850614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a3b1a-ec78-460b-8b34-90683e8bc06a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:06.880329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0772da2-b575-4c33-9551-3271227dcd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:06.888153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0772da2-b575-4c33-9551-3271227dcd38 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:06.888542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0772da2-b575-4c33-9551-3271227dcd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:06.889051 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0772da2-b575-4c33-9551-3271227dcd38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:06.892375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0772da2-b575-4c33-9551-3271227dcd38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:06.894263 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0772da2-b575-4c33-9551-3271227dcd38 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:06.940869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 851/1715] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:06.941507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0841e1c-6ecf-46b1-8fb7-a89d7e9722df service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:06.954695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0841e1c-6ecf-46b1-8fb7-a89d7e9722df service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:06.975331 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11c045-6622-4f0b-97e5-e10c9ad02c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:06.976037 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11c045-6622-4f0b-97e5-e10c9ad02c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:06.976314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11c045-6622-4f0b-97e5-e10c9ad02c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:06.976592 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11c045-6622-4f0b-97e5-e10c9ad02c4e admin admin] PATH_INFO: `/v3/projects/601b4970386147be930f13b9b590b223/users/e56ff1ab4e5b44b8b295a073979c1edd/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:07.033212 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11c045-6622-4f0b-97e5-e10c9ad02c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=601b4970386147be930f13b9b590b223, user_id=e56ff1ab4e5b44b8b295a073979c1edd, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:07.036093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11c045-6622-4f0b-97e5-e10c9ad02c4e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:07.068889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 865/1716] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:06 2022] PUT /identity/v3/projects/601b4970386147be930f13b9b590b223/users/e56ff1ab4e5b44b8b295a073979c1edd/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:07.085326 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d83731-636b-496e-a04f-13ffec3525b6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:07.085627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d83731-636b-496e-a04f-13ffec3525b6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:07.085934 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d83731-636b-496e-a04f-13ffec3525b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:07.197464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11c045-6622-4f0b-97e5-e10c9ad02c4e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:07.199611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1f11c045-6622-4f0b-97e5-e10c9ad02c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:07.200251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-59d83731-636b-496e-a04f-13ffec3525b6 None None] MFA Rules not processed for user `e56ff1ab4e5b44b8b295a073979c1edd`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:07.201518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-59d83731-636b-496e-a04f-13ffec3525b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:07.393741 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 852/1717] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:07.394777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d83731-636b-496e-a04f-13ffec3525b6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:07.444777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11c045-6622-4f0b-97e5-e10c9ad02c4e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:07.471819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f3620-a678-4b68-8add-7d6aae542c9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:07.472746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3620-a678-4b68-8add-7d6aae542c9b service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:07.473228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3620-a678-4b68-8add-7d6aae542c9b service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:07.473722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3620-a678-4b68-8add-7d6aae542c9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:07.489691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f3620-a678-4b68-8add-7d6aae542c9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:07.491635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f3620-a678-4b68-8add-7d6aae542c9b service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:07.546106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 866/1718] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:15:07.604525 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d83731-636b-496e-a04f-13ffec3525b6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:07.626494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af64cb-17f3-4962-948f-a1c4b9751150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:07.627285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af64cb-17f3-4962-948f-a1c4b9751150 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:07.627576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af64cb-17f3-4962-948f-a1c4b9751150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:07.627850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af64cb-17f3-4962-948f-a1c4b9751150 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:07.628574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af64cb-17f3-4962-948f-a1c4b9751150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:07.631815 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af64cb-17f3-4962-948f-a1c4b9751150 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:07.763917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 853/1719] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:07 2022] POST /identity/v3/projects => generated 377 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:07.765077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f3620-a678-4b68-8add-7d6aae542c9b service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:07.785073 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f3620-a678-4b68-8add-7d6aae542c9b service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:07.807823 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:07.968020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b7b58-9ea2-48be-8a64-b15d2dbaaa61 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:07.968555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b7b58-9ea2-48be-8a64-b15d2dbaaa61 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:07.968555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b7b58-9ea2-48be-8a64-b15d2dbaaa61 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:07.969923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 854/1720] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:15:07.992358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158bc8c-8fc4-4586-a061-1733428f8adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:07.993625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158bc8c-8fc4-4586-a061-1733428f8adc admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:07.997966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158bc8c-8fc4-4586-a061-1733428f8adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:07.998525 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158bc8c-8fc4-4586-a061-1733428f8adc admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:08.002668 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158bc8c-8fc4-4586-a061-1733428f8adc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:08.014238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158bc8c-8fc4-4586-a061-1733428f8adc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:08.115504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 867/1721] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:07 2022] POST /identity/v3/users => generated 295 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:08.116374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b7b58-9ea2-48be-8a64-b15d2dbaaa61 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:08.139468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b7b58-9ea2-48be-8a64-b15d2dbaaa61 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:08.149110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0640887-49ea-499a-ba67-935d0eeededf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:08.149782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0640887-49ea-499a-ba67-935d0eeededf admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:08.150059 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0640887-49ea-499a-ba67-935d0eeededf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:08.150336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0640887-49ea-499a-ba67-935d0eeededf admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:08.151006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0640887-49ea-499a-ba67-935d0eeededf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:08.159208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0640887-49ea-499a-ba67-935d0eeededf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:08.183459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 855/1722] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:08.184806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158bc8c-8fc4-4586-a061-1733428f8adc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:08.204595 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158bc8c-8fc4-4586-a061-1733428f8adc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:08.258314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0d9db-2d6b-42af-9f33-1ed5130d02f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:08.264715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0d9db-2d6b-42af-9f33-1ed5130d02f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:08.276258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0d9db-2d6b-42af-9f33-1ed5130d02f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:08.277450 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0d9db-2d6b-42af-9f33-1ed5130d02f2 admin admin] PATH_INFO: `/v3/projects/46e086801d224545b1fd4cdc0c61e8a9/users/c559fc41eb8a4409ba33d8e3ff9a2b8e/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:08.337481 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0d9db-2d6b-42af-9f33-1ed5130d02f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e086801d224545b1fd4cdc0c61e8a9, user_id=c559fc41eb8a4409ba33d8e3ff9a2b8e, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:08.341289 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0d9db-2d6b-42af-9f33-1ed5130d02f2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:08.388629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 868/1723] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:08 2022] PUT /identity/v3/projects/46e086801d224545b1fd4cdc0c61e8a9/users/c559fc41eb8a4409ba33d8e3ff9a2b8e/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:08.392894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0640887-49ea-499a-ba67-935d0eeededf admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:08.410909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0640887-49ea-499a-ba67-935d0eeededf admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:08.471172 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa886644-5b41-4f2a-9f4a-d6948cd5e68b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:08.471822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa886644-5b41-4f2a-9f4a-d6948cd5e68b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:08.472341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa886644-5b41-4f2a-9f4a-d6948cd5e68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:08.506129 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:08.520443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-fa886644-5b41-4f2a-9f4a-d6948cd5e68b None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:08.523563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fa886644-5b41-4f2a-9f4a-d6948cd5e68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:08.581281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631d246-9137-48a4-ac1a-5732d480e33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:08.584121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631d246-9137-48a4-ac1a-5732d480e33b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:08.584753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631d246-9137-48a4-ac1a-5732d480e33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:08.585332 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631d246-9137-48a4-ac1a-5732d480e33b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:08.588155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631d246-9137-48a4-ac1a-5732d480e33b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:08.602558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631d246-9137-48a4-ac1a-5732d480e33b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:08.642672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 856/1724] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:08.647219 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631d246-9137-48a4-ac1a-5732d480e33b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:08.681267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631d246-9137-48a4-ac1a-5732d480e33b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:08.697626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcf66d-6c61-4595-be43-271d0e626e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:08.698396 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcf66d-6c61-4595-be43-271d0e626e55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:08.698677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcf66d-6c61-4595-be43-271d0e626e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:08.698931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcf66d-6c61-4595-be43-271d0e626e55 admin admin] PATH_INFO: `/v3/projects/46e086801d224545b1fd4cdc0c61e8a9/users/c559fc41eb8a4409ba33d8e3ff9a2b8e/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:08.702161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcf66d-6c61-4595-be43-271d0e626e55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e086801d224545b1fd4cdc0c61e8a9, user_id=c559fc41eb8a4409ba33d8e3ff9a2b8e, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:08.705669 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcf66d-6c61-4595-be43-271d0e626e55 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:08.710409 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 869/1725] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:15:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:08.730732 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 857/1726] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:08 2022] PUT /identity/v3/projects/46e086801d224545b1fd4cdc0c61e8a9/users/c559fc41eb8a4409ba33d8e3ff9a2b8e/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:08.732962 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3b7e0-825a-4c3c-8693-e349dc016391 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:08.733343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3b7e0-825a-4c3c-8693-e349dc016391 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:08.737747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3b7e0-825a-4c3c-8693-e349dc016391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:08.782547 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b9d3b7e0-825a-4c3c-8693-e349dc016391 None None] MFA Rules not processed for user `c559fc41eb8a4409ba33d8e3ff9a2b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:08.784036 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b9d3b7e0-825a-4c3c-8693-e349dc016391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:08.820253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcf66d-6c61-4595-be43-271d0e626e55 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:08.823016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c0fcf66d-6c61-4595-be43-271d0e626e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:08.869161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:08.906533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 870/1727] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:08.922564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3b7e0-825a-4c3c-8693-e349dc016391 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:08.926264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b9d3b7e0-825a-4c3c-8693-e349dc016391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:09.115673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcf66d-6c61-4595-be43-271d0e626e55 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:09.147262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02092edc-41f2-4925-b945-bb01cedf18a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:09.147624 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02092edc-41f2-4925-b945-bb01cedf18a5 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:09.147795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02092edc-41f2-4925-b945-bb01cedf18a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:09.148029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02092edc-41f2-4925-b945-bb01cedf18a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:09.189310 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3b7e0-825a-4c3c-8693-e349dc016391 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:09.191146 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02092edc-41f2-4925-b945-bb01cedf18a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:09.196611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02092edc-41f2-4925-b945-bb01cedf18a5 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:09.200694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7093aed-62ab-4650-8285-c970f3ea453e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:09.202364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7093aed-62ab-4650-8285-c970f3ea453e service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:09.202988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7093aed-62ab-4650-8285-c970f3ea453e service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:09.204077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7093aed-62ab-4650-8285-c970f3ea453e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:09.256925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 858/1728] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:15:09.260347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2c250-e028-45d4-a55c-6bd6b74438b8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:09.260740 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2c250-e028-45d4-a55c-6bd6b74438b8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:09.261105 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2c250-e028-45d4-a55c-6bd6b74438b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:09.321809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-64f2c250-e028-45d4-a55c-6bd6b74438b8 None None] MFA Rules not processed for user `dcb88983a8e1439588644b71170d7362`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:09.325657 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-64f2c250-e028-45d4-a55c-6bd6b74438b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:09.348028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7093aed-62ab-4650-8285-c970f3ea453e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:09.353276 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7093aed-62ab-4650-8285-c970f3ea453e service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:09.415147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 871/1729] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:09.438781 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7093aed-62ab-4650-8285-c970f3ea453e service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:09.440581 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f7093aed-62ab-4650-8285-c970f3ea453e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:09.494098 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 859/1730] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:09.502840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2c250-e028-45d4-a55c-6bd6b74438b8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:09.506108 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-64f2c250-e028-45d4-a55c-6bd6b74438b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:09.630574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:09.703284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2c250-e028-45d4-a55c-6bd6b74438b8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:09.714124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7093aed-62ab-4650-8285-c970f3ea453e service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:09.722580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:09.726148 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:09.726554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:09.726899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] PATH_INFO: `/v3/projects/450f20a90a69450bab035066515b191e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:09.730284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] RBAC: Authorizing `identity:delete_project(project_id=450f20a90a69450bab035066515b191e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:09.739832 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef9eda-8b6a-48bd-bb4c-7036be00ffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:09.741361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef9eda-8b6a-48bd-bb4c-7036be00ffe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:09.741829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef9eda-8b6a-48bd-bb4c-7036be00ffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:09.741954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:09.742625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef9eda-8b6a-48bd-bb4c-7036be00ffe2 admin admin] PATH_INFO: `/v3/users/1ed72534f33f470993dc7e5406f84531` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:09.744247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef9eda-8b6a-48bd-bb4c-7036be00ffe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ed72534f33f470993dc7e5406f84531)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:09.747552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef9eda-8b6a-48bd-bb4c-7036be00ffe2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:09.869091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-12ef9eda-8b6a-48bd-bb4c-7036be00ffe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ed72534f33f470993dc7e5406f84531'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:09.870344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '450f20a90a69450bab035066515b191e', 'request_id': 'req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:09.872324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:09.893035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-12ef9eda-8b6a-48bd-bb4c-7036be00ffe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ed72534f33f470993dc7e5406f84531'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:09.894300 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-12ef9eda-8b6a-48bd-bb4c-7036be00ffe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ed72534f33f470993dc7e5406f84531'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:09.897226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '450f20a90a69450bab035066515b191e', 'request_id': 'req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:09.921873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-12ef9eda-8b6a-48bd-bb4c-7036be00ffe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ed72534f33f470993dc7e5406f84531'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:09.926401 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '450f20a90a69450bab035066515b191e', 'request_id': 'req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:09.943742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-12ef9eda-8b6a-48bd-bb4c-7036be00ffe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ed72534f33f470993dc7e5406f84531'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:09.963793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 872/1731] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:09 2022] DELETE /identity/v3/users/1ed72534f33f470993dc7e5406f84531 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:09.963793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2ed5a-0985-4192-9944-4df19a0c9716 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:09.963793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2ed5a-0985-4192-9944-4df19a0c9716 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:09.963793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2ed5a-0985-4192-9944-4df19a0c9716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:09.968202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] The token cache is being invalidate because project 450f20a90a69450bab035066515b191e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:09.971483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:09.972227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:09.979172 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 860/1732] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:09 2022] DELETE /identity/v3/projects/450f20a90a69450bab035066515b191e => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:09.980852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:09.983689 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:10.024330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e1e2ed5a-0985-4192-9944-4df19a0c9716 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:10.048558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e1e2ed5a-0985-4192-9944-4df19a0c9716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:10.175463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1b938-9994-4cf5-a7eb-e647c1b3019d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:10.318353 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:10.319128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:10.319507 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:10.319832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] PATH_INFO: `/v3/users/1b59460ac6244d19a597b681ce924c81` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:10.321189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b59460ac6244d19a597b681ce924c81)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:10.323425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:10.329673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 873/1733] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:10.375366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:10.471385 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b59460ac6244d19a597b681ce924c81'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:10.478247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b59460ac6244d19a597b681ce924c81'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:10.506268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2ed5a-0985-4192-9944-4df19a0c9716 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:10.510525 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e1e2ed5a-0985-4192-9944-4df19a0c9716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:10.519958 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b59460ac6244d19a597b681ce924c81'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:10.522856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b59460ac6244d19a597b681ce924c81'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:10.546260 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b59460ac6244d19a597b681ce924c81'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:10.560015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 861/1734] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:09 2022] DELETE /identity/v3/users/1b59460ac6244d19a597b681ce924c81 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:10.735750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2ed5a-0985-4192-9944-4df19a0c9716 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:10.907591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f82458-13c2-40e7-8bcb-54788bd965e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:10.908985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f82458-13c2-40e7-8bcb-54788bd965e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:10.909612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f82458-13c2-40e7-8bcb-54788bd965e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:10.910040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f82458-13c2-40e7-8bcb-54788bd965e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:10.914149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-08f82458-13c2-40e7-8bcb-54788bd965e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:11.010275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:11.089169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f82458-13c2-40e7-8bcb-54788bd965e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:11.090672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f82458-13c2-40e7-8bcb-54788bd965e8 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:11.130612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 874/1735] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:12.590217 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:12.593015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:12.620622 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c1df2-004c-4370-92c1-ed11fb8d40a3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:12.634758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:12.635548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:12.636694 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:12.636694 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] PATH_INFO: `/v3/projects/591239d3fe764bfcb6ef60ac30c25eed` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:12.640854 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=591239d3fe764bfcb6ef60ac30c25eed)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:12.643115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:12.761034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '591239d3fe764bfcb6ef60ac30c25eed', 'request_id': 'req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:12.778960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '591239d3fe764bfcb6ef60ac30c25eed', 'request_id': 'req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:12.795595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '591239d3fe764bfcb6ef60ac30c25eed', 'request_id': 'req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:12.820094 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] The token cache is being invalidate because project 591239d3fe764bfcb6ef60ac30c25eed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:12.820808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:12.821894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:12.823960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 862/1736] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:12 2022] DELETE /identity/v3/projects/591239d3fe764bfcb6ef60ac30c25eed => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:12.830136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd5703-4547-4e49-8ad4-61e67ea931ad None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:12.830575 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd5703-4547-4e49-8ad4-61e67ea931ad None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:12.830575 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd5703-4547-4e49-8ad4-61e67ea931ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:12.864597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1acd5703-4547-4e49-8ad4-61e67ea931ad None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:12.865864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1acd5703-4547-4e49-8ad4-61e67ea931ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:12.904303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:12.905882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:13.041296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 875/1737] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:13.052341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73ddea-fa7d-4235-b927-2071b1e151d0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:13.052844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73ddea-fa7d-4235-b927-2071b1e151d0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:13.053347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73ddea-fa7d-4235-b927-2071b1e151d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:13.065990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bb81f-7eaa-4f95-b029-694a04a8cc70 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:13.080611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:13.081543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:13.082123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:13.085321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] PATH_INFO: `/v3/projects/014fa4c673544a16b95c25d8af2cb70d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:13.088431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=014fa4c673544a16b95c25d8af2cb70d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:13.092103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:13.106691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9f73ddea-fa7d-4235-b927-2071b1e151d0 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:13.107986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9f73ddea-fa7d-4235-b927-2071b1e151d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:13.134581 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 876/1738] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 09 13:15:13.139943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73ddea-fa7d-4235-b927-2071b1e151d0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:13.157261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73ddea-fa7d-4235-b927-2071b1e151d0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:13.173907 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf346c-5792-4ad9-8cbb-dfee9399b879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:13.176077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf346c-5792-4ad9-8cbb-dfee9399b879 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:13.177906 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf346c-5792-4ad9-8cbb-dfee9399b879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:13.178770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf346c-5792-4ad9-8cbb-dfee9399b879 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:13.180648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf346c-5792-4ad9-8cbb-dfee9399b879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:15:13.181404 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf346c-5792-4ad9-8cbb-dfee9399b879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:13.183988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf346c-5792-4ad9-8cbb-dfee9399b879 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:13.188335 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:13.216120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 877/1739] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:15:13.219860 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '014fa4c673544a16b95c25d8af2cb70d', 'request_id': 'req-bbdf617b-7761-4fb9-9601-e57a01d77606'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:13.220537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf346c-5792-4ad9-8cbb-dfee9399b879 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:13.232195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf346c-5792-4ad9-8cbb-dfee9399b879 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:13.256578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '014fa4c673544a16b95c25d8af2cb70d', 'request_id': 'req-bbdf617b-7761-4fb9-9601-e57a01d77606'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:13.275677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '014fa4c673544a16b95c25d8af2cb70d', 'request_id': 'req-bbdf617b-7761-4fb9-9601-e57a01d77606'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:13.299109 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] The token cache is being invalidate because project 014fa4c673544a16b95c25d8af2cb70d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:13.299451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:13.300246 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:13.311731 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 863/1740] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:12 2022] DELETE /identity/v3/projects/014fa4c673544a16b95c25d8af2cb70d => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:13.321934 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:13.328293 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:13.370337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10440c08-11e2-4498-b446-1337abe32293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:13.371034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10440c08-11e2-4498-b446-1337abe32293 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:13.373346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10440c08-11e2-4498-b446-1337abe32293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:13.375271 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10440c08-11e2-4498-b446-1337abe32293 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:13.375749 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10440c08-11e2-4498-b446-1337abe32293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:13.382769 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10440c08-11e2-4498-b446-1337abe32293 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:13.487565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf617b-7761-4fb9-9601-e57a01d77606 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:13.492670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 878/1741] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:13 2022] POST /identity/v3/projects => generated 383 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:13.497515 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:13.498178 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:13.498462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:13.498879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10440c08-11e2-4498-b446-1337abe32293 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:13.499317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] PATH_INFO: `/v3/users/4a6e05c2a7fd4bad81d46ff6fe00cae0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:13.500635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-10440c08-11e2-4498-b446-1337abe32293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:13.500758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a6e05c2a7fd4bad81d46ff6fe00cae0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:13.503014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:13.574128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a6e05c2a7fd4bad81d46ff6fe00cae0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:13.579282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:13.586406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a6e05c2a7fd4bad81d46ff6fe00cae0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:13.616486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a6e05c2a7fd4bad81d46ff6fe00cae0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:13.617836 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a6e05c2a7fd4bad81d46ff6fe00cae0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:13.620858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10440c08-11e2-4498-b446-1337abe32293 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:13.627578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d84b8-480b-44f2-9a3f-cccbd3f1f72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:13.628179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a6e05c2a7fd4bad81d46ff6fe00cae0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:13.628356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d84b8-480b-44f2-9a3f-cccbd3f1f72f admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:13.628643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d84b8-480b-44f2-9a3f-cccbd3f1f72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:13.629058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d84b8-480b-44f2-9a3f-cccbd3f1f72f admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:13.629710 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d84b8-480b-44f2-9a3f-cccbd3f1f72f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:13.632052 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d84b8-480b-44f2-9a3f-cccbd3f1f72f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:13.640288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 864/1742] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:13 2022] DELETE /identity/v3/users/4a6e05c2a7fd4bad81d46ff6fe00cae0 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:13.671624 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 879/1743] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:13 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:13.673619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:13.675598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:13.738481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:13.760112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ace71-8275-4043-b504-0dd88374adb2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:13.765411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219dff5-5b81-4b12-bea0-560dffb1be7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:13.766037 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219dff5-5b81-4b12-bea0-560dffb1be7b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:13.766315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219dff5-5b81-4b12-bea0-560dffb1be7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:13.766584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219dff5-5b81-4b12-bea0-560dffb1be7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:13.767271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219dff5-5b81-4b12-bea0-560dffb1be7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:13.768884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219dff5-5b81-4b12-bea0-560dffb1be7b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:13.780286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 865/1744] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:13.784286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d84b8-480b-44f2-9a3f-cccbd3f1f72f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:13.794655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d84b8-480b-44f2-9a3f-cccbd3f1f72f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:13.800011 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76affc48-ed6d-40db-943f-99676a6d0bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:13.800866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76affc48-ed6d-40db-943f-99676a6d0bed admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:13.801200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76affc48-ed6d-40db-943f-99676a6d0bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:13.801495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76affc48-ed6d-40db-943f-99676a6d0bed admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:13.802219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76affc48-ed6d-40db-943f-99676a6d0bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:13.803971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76affc48-ed6d-40db-943f-99676a6d0bed admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:13.819963 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 880/1745] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:13.824325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219dff5-5b81-4b12-bea0-560dffb1be7b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:13.835693 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219dff5-5b81-4b12-bea0-560dffb1be7b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:13.841197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b3853-e6fa-40e3-93ea-7954128b0732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:13.841908 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b3853-e6fa-40e3-93ea-7954128b0732 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:13.842190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b3853-e6fa-40e3-93ea-7954128b0732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:13.842463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b3853-e6fa-40e3-93ea-7954128b0732 admin admin] PATH_INFO: `/v3/projects/e860c911879b418a8a28abbb67436755/users/915dd32f0fc2401585361ae1285f5351/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:13.858504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b3853-e6fa-40e3-93ea-7954128b0732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e860c911879b418a8a28abbb67436755, user_id=915dd32f0fc2401585361ae1285f5351, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:13.861255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b3853-e6fa-40e3-93ea-7954128b0732 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:13.876075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 866/1746] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:13 2022] PUT /identity/v3/projects/e860c911879b418a8a28abbb67436755/users/915dd32f0fc2401585361ae1285f5351/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:13.888153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2e6dc-4c4b-4118-be08-0028aefe367e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:13.888705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2e6dc-4c4b-4118-be08-0028aefe367e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:13.888803 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2e6dc-4c4b-4118-be08-0028aefe367e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:13.918926 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1ec2e6dc-4c4b-4118-be08-0028aefe367e None None] MFA Rules not processed for user `915dd32f0fc2401585361ae1285f5351`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:13.920284 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1ec2e6dc-4c4b-4118-be08-0028aefe367e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:13.939348 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b3853-e6fa-40e3-93ea-7954128b0732 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:13.942011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6b8b3853-e6fa-40e3-93ea-7954128b0732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:14.018570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 881/1747] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:14.021336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2e6dc-4c4b-4118-be08-0028aefe367e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.055752 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b3853-e6fa-40e3-93ea-7954128b0732 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.068769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c02777-ca9a-4c5f-9f20-05c820125247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.069463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c02777-ca9a-4c5f-9f20-05c820125247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.070164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c02777-ca9a-4c5f-9f20-05c820125247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.070409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c02777-ca9a-4c5f-9f20-05c820125247 admin admin] PATH_INFO: `/v3/projects/ae3e51af6f6644cc84a7872c7f926fcc` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.072860 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c02777-ca9a-4c5f-9f20-05c820125247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae3e51af6f6644cc84a7872c7f926fcc)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.075795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c02777-ca9a-4c5f-9f20-05c820125247 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.106120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2e6dc-4c4b-4118-be08-0028aefe367e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.112440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843e607-fd1e-4ed9-8e05-9c7810c7ffad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.113176 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843e607-fd1e-4ed9-8e05-9c7810c7ffad admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.113353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843e607-fd1e-4ed9-8e05-9c7810c7ffad admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.113629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843e607-fd1e-4ed9-8e05-9c7810c7ffad admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.114246 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843e607-fd1e-4ed9-8e05-9c7810c7ffad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.116324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843e607-fd1e-4ed9-8e05-9c7810c7ffad admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.130907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-73c02777-ca9a-4c5f-9f20-05c820125247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae3e51af6f6644cc84a7872c7f926fcc', 'request_id': 'req-73c02777-ca9a-4c5f-9f20-05c820125247'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:14.137216 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-73c02777-ca9a-4c5f-9f20-05c820125247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae3e51af6f6644cc84a7872c7f926fcc', 'request_id': 'req-73c02777-ca9a-4c5f-9f20-05c820125247'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:14.145511 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-73c02777-ca9a-4c5f-9f20-05c820125247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae3e51af6f6644cc84a7872c7f926fcc', 'request_id': 'req-73c02777-ca9a-4c5f-9f20-05c820125247'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:14.150226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 882/1748] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:14 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:14.154338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843e607-fd1e-4ed9-8e05-9c7810c7ffad admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.164040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-73c02777-ca9a-4c5f-9f20-05c820125247 admin admin] The token cache is being invalidate because project ae3e51af6f6644cc84a7872c7f926fcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:14.164166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-73c02777-ca9a-4c5f-9f20-05c820125247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:14.167470 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-73c02777-ca9a-4c5f-9f20-05c820125247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:14.173297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 867/1749] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:13 2022] DELETE /identity/v3/projects/ae3e51af6f6644cc84a7872c7f926fcc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:14.174679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c9866-7c09-48e3-86fc-bf0dfc05c5dc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.174953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c9866-7c09-48e3-86fc-bf0dfc05c5dc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.175283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c9866-7c09-48e3-86fc-bf0dfc05c5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.190365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:14.196035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-699c9866-7c09-48e3-86fc-bf0dfc05c5dc None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:14.197265 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-699c9866-7c09-48e3-86fc-bf0dfc05c5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:14.223123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843e607-fd1e-4ed9-8e05-9c7810c7ffad admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.227831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c25da5-4350-4f91-a3da-2f2721c348c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.228447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c25da5-4350-4f91-a3da-2f2721c348c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.228727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c25da5-4350-4f91-a3da-2f2721c348c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.228999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c25da5-4350-4f91-a3da-2f2721c348c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.229632 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c25da5-4350-4f91-a3da-2f2721c348c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.231957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c25da5-4350-4f91-a3da-2f2721c348c4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.244842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:14.258095 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 883/1750] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:14 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:14.261998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c25da5-4350-4f91-a3da-2f2721c348c4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.264549 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c8c25da5-4350-4f91-a3da-2f2721c348c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:14.276396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c25da5-4350-4f91-a3da-2f2721c348c4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.276749 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 868/1751] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:14.281349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c840101-ef46-4721-be6e-cfe2b75eecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.283153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c840101-ef46-4721-be6e-cfe2b75eecb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.283153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c840101-ef46-4721-be6e-cfe2b75eecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.283153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c840101-ef46-4721-be6e-cfe2b75eecb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.283786 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c840101-ef46-4721-be6e-cfe2b75eecb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.284174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08dba-ec5f-46d9-b68c-ad92fdebd6fd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.284462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08dba-ec5f-46d9-b68c-ad92fdebd6fd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.284738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08dba-ec5f-46d9-b68c-ad92fdebd6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.285394 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c840101-ef46-4721-be6e-cfe2b75eecb1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.297995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 884/1752] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:15:14.300075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c840101-ef46-4721-be6e-cfe2b75eecb1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.304307 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-19b08dba-ec5f-46d9-b68c-ad92fdebd6fd None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:14.305499 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-19b08dba-ec5f-46d9-b68c-ad92fdebd6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:14.310360 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c840101-ef46-4721-be6e-cfe2b75eecb1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.314348 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 869/1753] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:14.318102 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179f99c-da3f-4a6a-a147-5e889a6fc571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.318258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b08dba-ec5f-46d9-b68c-ad92fdebd6fd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.318751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179f99c-da3f-4a6a-a147-5e889a6fc571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.319027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179f99c-da3f-4a6a-a147-5e889a6fc571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.319333 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179f99c-da3f-4a6a-a147-5e889a6fc571 admin admin] PATH_INFO: `/v3/projects/5f361393903c4472b8cb7909b1731392/users/7e36d1c2cbe1451fb70072d312681bc1/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.327832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b08dba-ec5f-46d9-b68c-ad92fdebd6fd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.332974 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74086d-8c49-42e9-9f37-7be86af62c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.333568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74086d-8c49-42e9-9f37-7be86af62c32 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.333872 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74086d-8c49-42e9-9f37-7be86af62c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.334147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74086d-8c49-42e9-9f37-7be86af62c32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.334910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74086d-8c49-42e9-9f37-7be86af62c32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:15:14.335271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74086d-8c49-42e9-9f37-7be86af62c32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.335378 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179f99c-da3f-4a6a-a147-5e889a6fc571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f361393903c4472b8cb7909b1731392, user_id=7e36d1c2cbe1451fb70072d312681bc1, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.336860 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74086d-8c49-42e9-9f37-7be86af62c32 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.338019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179f99c-da3f-4a6a-a147-5e889a6fc571 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.349720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 870/1754] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:15:14.353222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 885/1755] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:14 2022] PUT /identity/v3/projects/5f361393903c4472b8cb7909b1731392/users/7e36d1c2cbe1451fb70072d312681bc1/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:14.355189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74086d-8c49-42e9-9f37-7be86af62c32 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.355349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179f99c-da3f-4a6a-a147-5e889a6fc571 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.365289 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74086d-8c49-42e9-9f37-7be86af62c32 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.421298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179f99c-da3f-4a6a-a147-5e889a6fc571 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.424274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcce459-05e2-4b65-a5ab-e0f06507dd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.424889 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcce459-05e2-4b65-a5ab-e0f06507dd23 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.425168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcce459-05e2-4b65-a5ab-e0f06507dd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.425494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcce459-05e2-4b65-a5ab-e0f06507dd23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.425936 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a17ab-3267-4adf-b3af-fd8e36417228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.426141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcce459-05e2-4b65-a5ab-e0f06507dd23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.426542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a17ab-3267-4adf-b3af-fd8e36417228 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.426821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a17ab-3267-4adf-b3af-fd8e36417228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.427122 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a17ab-3267-4adf-b3af-fd8e36417228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.427762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a17ab-3267-4adf-b3af-fd8e36417228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.428088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcce459-05e2-4b65-a5ab-e0f06507dd23 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.429329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a17ab-3267-4adf-b3af-fd8e36417228 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.442024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 886/1756] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:14.445990 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a17ab-3267-4adf-b3af-fd8e36417228 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.456709 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a17ab-3267-4adf-b3af-fd8e36417228 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.458055 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 871/1757] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:14 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:14.462114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcce459-05e2-4b65-a5ab-e0f06507dd23 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.520093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fed51-8ccc-4c7b-9541-36af39857b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.520763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fed51-8ccc-4c7b-9541-36af39857b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.521058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fed51-8ccc-4c7b-9541-36af39857b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.521312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fed51-8ccc-4c7b-9541-36af39857b68 admin admin] PATH_INFO: `/v3/projects/5f361393903c4472b8cb7909b1731392/users/7e36d1c2cbe1451fb70072d312681bc1/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.523869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4fed51-8ccc-4c7b-9541-36af39857b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f361393903c4472b8cb7909b1731392, user_id=7e36d1c2cbe1451fb70072d312681bc1, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.526706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4fed51-8ccc-4c7b-9541-36af39857b68 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.538048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 887/1758] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:14 2022] PUT /identity/v3/projects/5f361393903c4472b8cb7909b1731392/users/7e36d1c2cbe1451fb70072d312681bc1/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:14.541905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcce459-05e2-4b65-a5ab-e0f06507dd23 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.548646 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67a9f0-be26-472a-bd55-34c2916748a6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.548933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67a9f0-be26-472a-bd55-34c2916748a6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.549212 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67a9f0-be26-472a-bd55-34c2916748a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.579253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ed67a9f0-be26-472a-bd55-34c2916748a6 None None] MFA Rules not processed for user `7e36d1c2cbe1451fb70072d312681bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:14.580519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ed67a9f0-be26-472a-bd55-34c2916748a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:14.584836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:14.599334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:14.614244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca20da-ae39-40a9-a53e-c8ce88bdf4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.616619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca20da-ae39-40a9-a53e-c8ce88bdf4ff admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.617117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca20da-ae39-40a9-a53e-c8ce88bdf4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.617455 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca20da-ae39-40a9-a53e-c8ce88bdf4ff admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.619267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca20da-ae39-40a9-a53e-c8ce88bdf4ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.621527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca20da-ae39-40a9-a53e-c8ce88bdf4ff admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.649997 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 872/1759] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:14 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:14.651084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca20da-ae39-40a9-a53e-c8ce88bdf4ff admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.664068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca20da-ae39-40a9-a53e-c8ce88bdf4ff admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.667319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 888/1760] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:14.672863 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226645e7-3aaa-427f-b1f7-6d983c976ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.673733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226645e7-3aaa-427f-b1f7-6d983c976ffc admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.674234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226645e7-3aaa-427f-b1f7-6d983c976ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.674710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226645e7-3aaa-427f-b1f7-6d983c976ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.675672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226645e7-3aaa-427f-b1f7-6d983c976ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.677497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226645e7-3aaa-427f-b1f7-6d983c976ffc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.689535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 873/1761] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:14.690912 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67a9f0-be26-472a-bd55-34c2916748a6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.692616 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ed67a9f0-be26-472a-bd55-34c2916748a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:14.693283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226645e7-3aaa-427f-b1f7-6d983c976ffc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.705890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226645e7-3aaa-427f-b1f7-6d983c976ffc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.714017 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a26794-eafb-4cde-9e13-41066f98d802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.714736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a26794-eafb-4cde-9e13-41066f98d802 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.715107 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a26794-eafb-4cde-9e13-41066f98d802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.715576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a26794-eafb-4cde-9e13-41066f98d802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.716303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a26794-eafb-4cde-9e13-41066f98d802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.717906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a26794-eafb-4cde-9e13-41066f98d802 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.728817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 874/1762] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:15:14.733606 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a26794-eafb-4cde-9e13-41066f98d802 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.743318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a26794-eafb-4cde-9e13-41066f98d802 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.750604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85de02-0d6f-46a0-9a8f-c44ce6624b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.751310 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85de02-0d6f-46a0-9a8f-c44ce6624b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.751590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85de02-0d6f-46a0-9a8f-c44ce6624b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.751986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85de02-0d6f-46a0-9a8f-c44ce6624b03 admin admin] PATH_INFO: `/v3/projects/ba92353c2f41468b884befaec6d4feb3/users/3803f3745fb44ab089c4853c5f9821f6/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.757562 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67a9f0-be26-472a-bd55-34c2916748a6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.761645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b409d2-b09e-4800-af76-212e5d5e2b19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.762258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b409d2-b09e-4800-af76-212e5d5e2b19 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.762544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b409d2-b09e-4800-af76-212e5d5e2b19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.762818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b409d2-b09e-4800-af76-212e5d5e2b19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.769278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85de02-0d6f-46a0-9a8f-c44ce6624b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba92353c2f41468b884befaec6d4feb3, user_id=3803f3745fb44ab089c4853c5f9821f6, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.773329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85de02-0d6f-46a0-9a8f-c44ce6624b03 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.773930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b409d2-b09e-4800-af76-212e5d5e2b19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:14.775405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b409d2-b09e-4800-af76-212e5d5e2b19 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:14.786388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 889/1763] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:14.795501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 875/1764] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:14 2022] PUT /identity/v3/projects/ba92353c2f41468b884befaec6d4feb3/users/3803f3745fb44ab089c4853c5f9821f6/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:14.807114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88074294-55d8-46ef-86fb-992ea7a20851 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.807114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88074294-55d8-46ef-86fb-992ea7a20851 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.807114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88074294-55d8-46ef-86fb-992ea7a20851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.840184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-88074294-55d8-46ef-86fb-992ea7a20851 None None] MFA Rules not processed for user `3803f3745fb44ab089c4853c5f9821f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:14.843608 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-88074294-55d8-46ef-86fb-992ea7a20851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:14.909680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85de02-0d6f-46a0-9a8f-c44ce6624b03 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.911493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ce85de02-0d6f-46a0-9a8f-c44ce6624b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:14.935204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 890/1765] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:14.937500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88074294-55d8-46ef-86fb-992ea7a20851 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:14.978454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:14.985055 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85de02-0d6f-46a0-9a8f-c44ce6624b03 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:14.991619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714526e2-e7de-400e-8dbc-51a78e453e27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:14.992253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714526e2-e7de-400e-8dbc-51a78e453e27 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:14.992534 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714526e2-e7de-400e-8dbc-51a78e453e27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:14.992828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714526e2-e7de-400e-8dbc-51a78e453e27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:14.994608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-714526e2-e7de-400e-8dbc-51a78e453e27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:15.018320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88074294-55d8-46ef-86fb-992ea7a20851 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:15.023523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11e094-8740-4d8d-8fd3-08a2ec08af87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:15.024020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11e094-8740-4d8d-8fd3-08a2ec08af87 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:15.024326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11e094-8740-4d8d-8fd3-08a2ec08af87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:15.024600 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11e094-8740-4d8d-8fd3-08a2ec08af87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:15.025234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11e094-8740-4d8d-8fd3-08a2ec08af87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:15.027208 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11e094-8740-4d8d-8fd3-08a2ec08af87 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:15.063451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 891/1766] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:14 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:15.070272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714526e2-e7de-400e-8dbc-51a78e453e27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:15.071132 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11e094-8740-4d8d-8fd3-08a2ec08af87 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:15.071823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714526e2-e7de-400e-8dbc-51a78e453e27 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:15.088057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 876/1767] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:15.142625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11e094-8740-4d8d-8fd3-08a2ec08af87 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:15.147486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4214b-2f7a-4a35-87ac-8900d0ca64f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:15.148106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4214b-2f7a-4a35-87ac-8900d0ca64f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:15.148388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4214b-2f7a-4a35-87ac-8900d0ca64f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:15.148660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4214b-2f7a-4a35-87ac-8900d0ca64f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:15.149323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4214b-2f7a-4a35-87ac-8900d0ca64f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:15.151282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4214b-2f7a-4a35-87ac-8900d0ca64f9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:15.178702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 892/1768] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:15 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:15.180269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714526e2-e7de-400e-8dbc-51a78e453e27 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:15.191276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714526e2-e7de-400e-8dbc-51a78e453e27 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:15.195835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e00d08-f43b-4ece-8a36-b476ea534b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:15.196439 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e00d08-f43b-4ece-8a36-b476ea534b85 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:15.196714 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e00d08-f43b-4ece-8a36-b476ea534b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:15.196981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e00d08-f43b-4ece-8a36-b476ea534b85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:15.197616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e00d08-f43b-4ece-8a36-b476ea534b85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:15.199242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e00d08-f43b-4ece-8a36-b476ea534b85 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:15.209271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 877/1769] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:15.213380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4214b-2f7a-4a35-87ac-8900d0ca64f9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:15.223985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4214b-2f7a-4a35-87ac-8900d0ca64f9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:15.228761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868252ab-d857-4f72-8991-4fa66ab7c435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:15.229412 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868252ab-d857-4f72-8991-4fa66ab7c435 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:15.229691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868252ab-d857-4f72-8991-4fa66ab7c435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:15.229966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868252ab-d857-4f72-8991-4fa66ab7c435 admin admin] PATH_INFO: `/v3/projects/1481fd4b1b3041b1956708ac5aa391d3/users/74cd871b885f4df187e4830e4fc249fb/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:15.248096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868252ab-d857-4f72-8991-4fa66ab7c435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1481fd4b1b3041b1956708ac5aa391d3, user_id=74cd871b885f4df187e4830e4fc249fb, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:15.250962 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868252ab-d857-4f72-8991-4fa66ab7c435 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:15.268730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 893/1770] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:15 2022] PUT /identity/v3/projects/1481fd4b1b3041b1956708ac5aa391d3/users/74cd871b885f4df187e4830e4fc249fb/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:15.272779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e00d08-f43b-4ece-8a36-b476ea534b85 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:15.284097 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e00d08-f43b-4ece-8a36-b476ea534b85 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:15.330052 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:15.359770 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918eead-5e9d-4da0-9ce2-5adb44401549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:15.360584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918eead-5e9d-4da0-9ce2-5adb44401549 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:15.360991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918eead-5e9d-4da0-9ce2-5adb44401549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:15.361488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918eead-5e9d-4da0-9ce2-5adb44401549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:15.362601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918eead-5e9d-4da0-9ce2-5adb44401549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:15.364588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918eead-5e9d-4da0-9ce2-5adb44401549 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:15.375843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 878/1771] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:15.380037 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868252ab-d857-4f72-8991-4fa66ab7c435 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:15.391873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868252ab-d857-4f72-8991-4fa66ab7c435 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:15.396766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f02b78-2832-48bd-a089-b2a742bb9446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:15.397341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f02b78-2832-48bd-a089-b2a742bb9446 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:15.397616 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f02b78-2832-48bd-a089-b2a742bb9446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:15.397887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f02b78-2832-48bd-a089-b2a742bb9446 admin admin] PATH_INFO: `/v3/projects/1481fd4b1b3041b1956708ac5aa391d3/users/74cd871b885f4df187e4830e4fc249fb/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:15.399856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f02b78-2832-48bd-a089-b2a742bb9446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1481fd4b1b3041b1956708ac5aa391d3, user_id=74cd871b885f4df187e4830e4fc249fb, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:15.402474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f02b78-2832-48bd-a089-b2a742bb9446 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:15.419162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 894/1772] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:15 2022] PUT /identity/v3/projects/1481fd4b1b3041b1956708ac5aa391d3/users/74cd871b885f4df187e4830e4fc249fb/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:15.428584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e82b8-8506-4a47-b39e-e5102e1f6fe3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:15.428939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e82b8-8506-4a47-b39e-e5102e1f6fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:15.429292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e82b8-8506-4a47-b39e-e5102e1f6fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:15.469660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0a4e82b8-8506-4a47-b39e-e5102e1f6fe3 None None] MFA Rules not processed for user `74cd871b885f4df187e4830e4fc249fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:15.471129 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0a4e82b8-8506-4a47-b39e-e5102e1f6fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:15.544724 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 879/1773] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:15 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:15.556090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f02b78-2832-48bd-a089-b2a742bb9446 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:15.612407 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f02b78-2832-48bd-a089-b2a742bb9446 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:15.616290 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308635a-88dd-4258-b0c4-d81800b2b148 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:15.616946 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308635a-88dd-4258-b0c4-d81800b2b148 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:15.617253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308635a-88dd-4258-b0c4-d81800b2b148 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:15.617530 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308635a-88dd-4258-b0c4-d81800b2b148 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:15.627330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308635a-88dd-4258-b0c4-d81800b2b148 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:15.628756 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308635a-88dd-4258-b0c4-d81800b2b148 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:15.638491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 895/1774] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:15 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:15.667807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e82b8-8506-4a47-b39e-e5102e1f6fe3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:15.733896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e82b8-8506-4a47-b39e-e5102e1f6fe3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:15.738795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:15.739482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:15.739764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:15.740038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] PATH_INFO: `/v3/users/3803f3745fb44ab089c4853c5f9821f6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:15.741203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3803f3745fb44ab089c4853c5f9821f6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:15.743172 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:15.793950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3803f3745fb44ab089c4853c5f9821f6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:15.802261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3803f3745fb44ab089c4853c5f9821f6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:15.810446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:15.817264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3803f3745fb44ab089c4853c5f9821f6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:15.818159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3803f3745fb44ab089c4853c5f9821f6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:15.826152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3803f3745fb44ab089c4853c5f9821f6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:15.832240 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 880/1775] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:15 2022] DELETE /identity/v3/users/3803f3745fb44ab089c4853c5f9821f6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:15.836658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d37ff6-34b3-42f1-89eb-d1373850697c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:15.836953 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d37ff6-34b3-42f1-89eb-d1373850697c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:15.837237 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d37ff6-34b3-42f1-89eb-d1373850697c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:15.857463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-88d37ff6-34b3-42f1-89eb-d1373850697c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:15.858673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-88d37ff6-34b3-42f1-89eb-d1373850697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:15.867180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:15.922978 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 896/1776] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:15.932699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:15.934666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:15.992719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97d7f0-fc8b-413c-a2c7-13ca67effcb1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:15.996487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479c4a4-e178-4522-82da-441f60abdf29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:15.997297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479c4a4-e178-4522-82da-441f60abdf29 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:15.997587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479c4a4-e178-4522-82da-441f60abdf29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:15.997861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479c4a4-e178-4522-82da-441f60abdf29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:16.006909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479c4a4-e178-4522-82da-441f60abdf29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:16.008394 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479c4a4-e178-4522-82da-441f60abdf29 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:16.020046 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 881/1777] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:16.308424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d37ff6-34b3-42f1-89eb-d1373850697c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:16.310041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-88d37ff6-34b3-42f1-89eb-d1373850697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:16.325636 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d37ff6-34b3-42f1-89eb-d1373850697c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:16.333847 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:16.334704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:16.334993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:16.335298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] PATH_INFO: `/v3/projects/ba92353c2f41468b884befaec6d4feb3` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:16.336440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba92353c2f41468b884befaec6d4feb3)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:16.338339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:16.386136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba92353c2f41468b884befaec6d4feb3', 'request_id': 'req-dafee397-924b-4bc5-acdb-706f0c081f68'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:16.394337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba92353c2f41468b884befaec6d4feb3', 'request_id': 'req-dafee397-924b-4bc5-acdb-706f0c081f68'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:16.400133 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba92353c2f41468b884befaec6d4feb3', 'request_id': 'req-dafee397-924b-4bc5-acdb-706f0c081f68'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:16.411357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] The token cache is being invalidate because project ba92353c2f41468b884befaec6d4feb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:16.411642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:16.412269 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:16.413757 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 897/1778] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:16 2022] DELETE /identity/v3/projects/ba92353c2f41468b884befaec6d4feb3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:16.417852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479c4a4-e178-4522-82da-441f60abdf29 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:16.427534 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479c4a4-e178-4522-82da-441f60abdf29 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:16.493016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218dd324-4550-4dad-90d1-41be377e23aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:16.493708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218dd324-4550-4dad-90d1-41be377e23aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:16.493987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218dd324-4550-4dad-90d1-41be377e23aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:16.494257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218dd324-4550-4dad-90d1-41be377e23aa admin admin] PATH_INFO: `/v3/users/74cd871b885f4df187e4830e4fc249fb` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:16.495417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218dd324-4550-4dad-90d1-41be377e23aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=74cd871b885f4df187e4830e4fc249fb)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:16.498408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218dd324-4550-4dad-90d1-41be377e23aa admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:16.546720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-218dd324-4550-4dad-90d1-41be377e23aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74cd871b885f4df187e4830e4fc249fb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:16.555769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-218dd324-4550-4dad-90d1-41be377e23aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74cd871b885f4df187e4830e4fc249fb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:16.572316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-218dd324-4550-4dad-90d1-41be377e23aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74cd871b885f4df187e4830e4fc249fb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:16.573321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-218dd324-4550-4dad-90d1-41be377e23aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74cd871b885f4df187e4830e4fc249fb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:16.583169 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-218dd324-4550-4dad-90d1-41be377e23aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74cd871b885f4df187e4830e4fc249fb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:16.589527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 882/1779] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:16 2022] DELETE /identity/v3/users/74cd871b885f4df187e4830e4fc249fb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:16.895573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:16.897504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:16.921201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee397-924b-4bc5-acdb-706f0c081f68 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:16.985908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51ae1d-e4f2-4271-acf0-1867031de0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:16.986543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51ae1d-e4f2-4271-acf0-1867031de0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:16.987113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51ae1d-e4f2-4271-acf0-1867031de0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:16.987403 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51ae1d-e4f2-4271-acf0-1867031de0e2 admin admin] PATH_INFO: `/v3/projects/1481fd4b1b3041b1956708ac5aa391d3` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:16.988499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51ae1d-e4f2-4271-acf0-1867031de0e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1481fd4b1b3041b1956708ac5aa391d3)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:16.990403 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51ae1d-e4f2-4271-acf0-1867031de0e2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:17.032953 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:17.043355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7f51ae1d-e4f2-4271-acf0-1867031de0e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1481fd4b1b3041b1956708ac5aa391d3', 'request_id': 'req-7f51ae1d-e4f2-4271-acf0-1867031de0e2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:17.050893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7f51ae1d-e4f2-4271-acf0-1867031de0e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1481fd4b1b3041b1956708ac5aa391d3', 'request_id': 'req-7f51ae1d-e4f2-4271-acf0-1867031de0e2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:17.056824 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7f51ae1d-e4f2-4271-acf0-1867031de0e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1481fd4b1b3041b1956708ac5aa391d3', 'request_id': 'req-7f51ae1d-e4f2-4271-acf0-1867031de0e2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:17.068506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7f51ae1d-e4f2-4271-acf0-1867031de0e2 admin admin] The token cache is being invalidate because project 1481fd4b1b3041b1956708ac5aa391d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:17.068786 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7f51ae1d-e4f2-4271-acf0-1867031de0e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:17.069425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7f51ae1d-e4f2-4271-acf0-1867031de0e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:17.070889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 898/1780] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:16 2022] DELETE /identity/v3/projects/1481fd4b1b3041b1956708ac5aa391d3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:17.076237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d904ae8-db3e-4cae-b833-98e6ddf5d992 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:17.076643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d904ae8-db3e-4cae-b833-98e6ddf5d992 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:17.076819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d904ae8-db3e-4cae-b833-98e6ddf5d992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:17.084449 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:17.099482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4d904ae8-db3e-4cae-b833-98e6ddf5d992 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:17.100700 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4d904ae8-db3e-4cae-b833-98e6ddf5d992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:17.159702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 883/1781] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:17.167228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e77153-a9d4-4463-a9bd-187808a0f1cb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:17.167515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e77153-a9d4-4463-a9bd-187808a0f1cb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:17.167783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e77153-a9d4-4463-a9bd-187808a0f1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:17.187514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-16e77153-a9d4-4463-a9bd-187808a0f1cb None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:17.188704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-16e77153-a9d4-4463-a9bd-187808a0f1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:17.196431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 899/1782] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:17.201745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d904ae8-db3e-4cae-b833-98e6ddf5d992 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:17.211636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d904ae8-db3e-4cae-b833-98e6ddf5d992 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:17.216392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b7fbc-0a66-4368-a9e6-3a277a636cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:17.216990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b7fbc-0a66-4368-a9e6-3a277a636cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:17.217266 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b7fbc-0a66-4368-a9e6-3a277a636cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:17.217536 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b7fbc-0a66-4368-a9e6-3a277a636cb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:17.218179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b7fbc-0a66-4368-a9e6-3a277a636cb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:15:17.218480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b7fbc-0a66-4368-a9e6-3a277a636cb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:17.220099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b7fbc-0a66-4368-a9e6-3a277a636cb4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:17.232273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 884/1783] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:15:17.236753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e77153-a9d4-4463-a9bd-187808a0f1cb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:17.246526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e77153-a9d4-4463-a9bd-187808a0f1cb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:17.252112 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487ae52-e6d5-496b-9bb9-6f4758b88bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:17.252822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487ae52-e6d5-496b-9bb9-6f4758b88bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:17.253222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487ae52-e6d5-496b-9bb9-6f4758b88bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:17.253501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487ae52-e6d5-496b-9bb9-6f4758b88bf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:17.254169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487ae52-e6d5-496b-9bb9-6f4758b88bf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:17.256154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487ae52-e6d5-496b-9bb9-6f4758b88bf2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:17.287419 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 900/1784] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:17 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:17.291718 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b7fbc-0a66-4368-a9e6-3a277a636cb4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:17.311631 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b7fbc-0a66-4368-a9e6-3a277a636cb4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:17.378879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bdd4d-e580-4881-a6ca-edf6e354789a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:17.379539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bdd4d-e580-4881-a6ca-edf6e354789a admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:17.379816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bdd4d-e580-4881-a6ca-edf6e354789a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:17.380087 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bdd4d-e580-4881-a6ca-edf6e354789a admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:17.380750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bdd4d-e580-4881-a6ca-edf6e354789a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:17.382734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bdd4d-e580-4881-a6ca-edf6e354789a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:17.408730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 885/1785] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:17 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:17.409927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487ae52-e6d5-496b-9bb9-6f4758b88bf2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:17.420420 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487ae52-e6d5-496b-9bb9-6f4758b88bf2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:17.425045 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da3252-22de-4496-a80b-f4bc7a166a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:17.425666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da3252-22de-4496-a80b-f4bc7a166a93 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:17.425947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da3252-22de-4496-a80b-f4bc7a166a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:17.426220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da3252-22de-4496-a80b-f4bc7a166a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:17.426872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da3252-22de-4496-a80b-f4bc7a166a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:17.428502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da3252-22de-4496-a80b-f4bc7a166a93 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:17.438825 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 901/1786] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:17.442624 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bdd4d-e580-4881-a6ca-edf6e354789a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:17.453056 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bdd4d-e580-4881-a6ca-edf6e354789a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:17.457752 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8fb1c-5512-4d15-a813-cf15f7103485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:17.458354 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8fb1c-5512-4d15-a813-cf15f7103485 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:17.458635 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8fb1c-5512-4d15-a813-cf15f7103485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:17.458910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8fb1c-5512-4d15-a813-cf15f7103485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:17.459601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8fb1c-5512-4d15-a813-cf15f7103485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:17.461552 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8fb1c-5512-4d15-a813-cf15f7103485 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:17.472149 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 886/1787] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:17.476338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da3252-22de-4496-a80b-f4bc7a166a93 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:17.486713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da3252-22de-4496-a80b-f4bc7a166a93 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:17.491623 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9aa84-8654-4ae5-9e9f-f2aabc04655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:17.492266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9aa84-8654-4ae5-9e9f-f2aabc04655c admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:17.492547 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9aa84-8654-4ae5-9e9f-f2aabc04655c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:17.492822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9aa84-8654-4ae5-9e9f-f2aabc04655c admin admin] PATH_INFO: `/v3/projects/69948db1d7a546708fb9f07b9e047722/users/1505cde569dc4edfb7c6ff8ae34bcd52/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:17.508254 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9aa84-8654-4ae5-9e9f-f2aabc04655c admin admin] RBAC: Authorizing `identity:create_grant(project_id=69948db1d7a546708fb9f07b9e047722, user_id=1505cde569dc4edfb7c6ff8ae34bcd52, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:17.510909 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9aa84-8654-4ae5-9e9f-f2aabc04655c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:17.524820 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 902/1788] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:17 2022] PUT /identity/v3/projects/69948db1d7a546708fb9f07b9e047722/users/1505cde569dc4edfb7c6ff8ae34bcd52/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:17.535183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d0789-4323-4e6e-a84a-da86bb17b01d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:17.535474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d0789-4323-4e6e-a84a-da86bb17b01d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:17.535753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d0789-4323-4e6e-a84a-da86bb17b01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:17.568249 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ed1d0789-4323-4e6e-a84a-da86bb17b01d None None] MFA Rules not processed for user `1505cde569dc4edfb7c6ff8ae34bcd52`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:17.569468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ed1d0789-4323-4e6e-a84a-da86bb17b01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:17.631346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:17.641711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 887/1789] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:17.642905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9aa84-8654-4ae5-9e9f-f2aabc04655c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:17.731468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9aa84-8654-4ae5-9e9f-f2aabc04655c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:17.750001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f25394-e8b4-4596-bd93-397fb1fc0e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:17.750870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25394-e8b4-4596-bd93-397fb1fc0e15 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:17.751166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25394-e8b4-4596-bd93-397fb1fc0e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:17.751485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25394-e8b4-4596-bd93-397fb1fc0e15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:17.752687 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f25394-e8b4-4596-bd93-397fb1fc0e15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:17.756231 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f25394-e8b4-4596-bd93-397fb1fc0e15 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:17.792282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 903/1790] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:17 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:17.797085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d0789-4323-4e6e-a84a-da86bb17b01d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:17.810704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d0789-4323-4e6e-a84a-da86bb17b01d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:17.886467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1fcbb-0a83-4514-ba69-c438ca08dcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:17.887120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1fcbb-0a83-4514-ba69-c438ca08dcfb admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:17.887481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1fcbb-0a83-4514-ba69-c438ca08dcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:17.887776 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1fcbb-0a83-4514-ba69-c438ca08dcfb admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:17.889091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1fcbb-0a83-4514-ba69-c438ca08dcfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:17.891316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1fcbb-0a83-4514-ba69-c438ca08dcfb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:17.913872 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 888/1791] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:17 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:17.918009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f25394-e8b4-4596-bd93-397fb1fc0e15 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:17.928595 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f25394-e8b4-4596-bd93-397fb1fc0e15 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:17.933309 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7ac3d-c434-4df9-a271-b08fe07973a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:17.933938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7ac3d-c434-4df9-a271-b08fe07973a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:17.934751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7ac3d-c434-4df9-a271-b08fe07973a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:17.935028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7ac3d-c434-4df9-a271-b08fe07973a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:17.935718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7ac3d-c434-4df9-a271-b08fe07973a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:17.937397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7ac3d-c434-4df9-a271-b08fe07973a4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:17.951888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 904/1792] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:17.953088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1fcbb-0a83-4514-ba69-c438ca08dcfb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:17.963652 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1fcbb-0a83-4514-ba69-c438ca08dcfb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:17.968614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca44da-9dad-4d45-9379-32c31a8ea729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:17.969227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca44da-9dad-4d45-9379-32c31a8ea729 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:17.969509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca44da-9dad-4d45-9379-32c31a8ea729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:17.969785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca44da-9dad-4d45-9379-32c31a8ea729 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:17.970429 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca44da-9dad-4d45-9379-32c31a8ea729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:17.972066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca44da-9dad-4d45-9379-32c31a8ea729 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:17.982305 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 889/1793] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:17.986363 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7ac3d-c434-4df9-a271-b08fe07973a4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:17.996532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7ac3d-c434-4df9-a271-b08fe07973a4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:18.001923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cfb7a-7c75-4562-a792-a069cd319bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:18.002567 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cfb7a-7c75-4562-a792-a069cd319bec admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:18.002843 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cfb7a-7c75-4562-a792-a069cd319bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:18.003217 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cfb7a-7c75-4562-a792-a069cd319bec admin admin] PATH_INFO: `/v3/projects/f2d566baeace4244869c3c7a134188ca/users/8ecb4007b2024b31b7d6f04c16ccf66a/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:18.018149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cfb7a-7c75-4562-a792-a069cd319bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d566baeace4244869c3c7a134188ca, user_id=8ecb4007b2024b31b7d6f04c16ccf66a, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:18.020815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cfb7a-7c75-4562-a792-a069cd319bec admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:18.033848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 905/1794] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:17 2022] PUT /identity/v3/projects/f2d566baeace4244869c3c7a134188ca/users/8ecb4007b2024b31b7d6f04c16ccf66a/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:18.045514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafb8f0-1f82-41fc-a257-11d529fa2844 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:18.045801 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafb8f0-1f82-41fc-a257-11d529fa2844 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:18.046071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafb8f0-1f82-41fc-a257-11d529fa2844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:18.078195 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ddafb8f0-1f82-41fc-a257-11d529fa2844 None None] MFA Rules not processed for user `8ecb4007b2024b31b7d6f04c16ccf66a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:18.079518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ddafb8f0-1f82-41fc-a257-11d529fa2844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:18.146110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 890/1795] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:18.151608 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cfb7a-7c75-4562-a792-a069cd319bec admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:18.169909 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:18.176652 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafb8f0-1f82-41fc-a257-11d529fa2844 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:18.178135 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ddafb8f0-1f82-41fc-a257-11d529fa2844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:18.233802 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cfb7a-7c75-4562-a792-a069cd319bec admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:18.239009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71102e80-1371-42a3-8e78-42458bedda1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:18.239648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71102e80-1371-42a3-8e78-42458bedda1d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:18.239929 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71102e80-1371-42a3-8e78-42458bedda1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:18.240202 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71102e80-1371-42a3-8e78-42458bedda1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:18.240846 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71102e80-1371-42a3-8e78-42458bedda1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:18.242734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71102e80-1371-42a3-8e78-42458bedda1d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:18.250401 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafb8f0-1f82-41fc-a257-11d529fa2844 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:18.253984 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd6c06-e1e0-4c54-9184-17ba70e83702 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:18.254588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd6c06-e1e0-4c54-9184-17ba70e83702 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:18.254864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd6c06-e1e0-4c54-9184-17ba70e83702 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:18.255173 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd6c06-e1e0-4c54-9184-17ba70e83702 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:18.256928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4ecd6c06-e1e0-4c54-9184-17ba70e83702 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:18.271107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 906/1796] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:18 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:18.275146 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71102e80-1371-42a3-8e78-42458bedda1d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:18.278205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:18.324175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd6c06-e1e0-4c54-9184-17ba70e83702 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:18.325864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd6c06-e1e0-4c54-9184-17ba70e83702 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:18.336453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 891/1797] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:15:18.337381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71102e80-1371-42a3-8e78-42458bedda1d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:18.342312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f358cdf-b0cb-46f1-9380-fdcd0d4d369e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:18.342922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f358cdf-b0cb-46f1-9380-fdcd0d4d369e admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:18.343236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f358cdf-b0cb-46f1-9380-fdcd0d4d369e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:18.343515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f358cdf-b0cb-46f1-9380-fdcd0d4d369e admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:18.344140 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f358cdf-b0cb-46f1-9380-fdcd0d4d369e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:18.346038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f358cdf-b0cb-46f1-9380-fdcd0d4d369e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:18.376984 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 907/1798] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:18 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:18.378499 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd6c06-e1e0-4c54-9184-17ba70e83702 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:18.395713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd6c06-e1e0-4c54-9184-17ba70e83702 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:18.401155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbecf4-e847-41a5-aa82-0a31541d2858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:18.401887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbecf4-e847-41a5-aa82-0a31541d2858 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:18.402174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbecf4-e847-41a5-aa82-0a31541d2858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:18.402453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbecf4-e847-41a5-aa82-0a31541d2858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:18.403525 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbecf4-e847-41a5-aa82-0a31541d2858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:18.405278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbecf4-e847-41a5-aa82-0a31541d2858 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:18.417215 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 892/1799] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:18.422377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f358cdf-b0cb-46f1-9380-fdcd0d4d369e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:18.433773 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f358cdf-b0cb-46f1-9380-fdcd0d4d369e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:18.439368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489146fb-5585-4517-8c23-a9c4b5768627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:18.440162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489146fb-5585-4517-8c23-a9c4b5768627 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:18.440450 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489146fb-5585-4517-8c23-a9c4b5768627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:18.440727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489146fb-5585-4517-8c23-a9c4b5768627 admin admin] PATH_INFO: `/v3/projects/2d9a2d86417b4395ad94728e6e879ba8/users/3bba5acc965047579ed14b6700949f48/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:18.474263 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489146fb-5585-4517-8c23-a9c4b5768627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d9a2d86417b4395ad94728e6e879ba8, user_id=3bba5acc965047579ed14b6700949f48, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:18.477107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489146fb-5585-4517-8c23-a9c4b5768627 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:18.496628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 908/1800] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:18 2022] PUT /identity/v3/projects/2d9a2d86417b4395ad94728e6e879ba8/users/3bba5acc965047579ed14b6700949f48/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:18.501011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbecf4-e847-41a5-aa82-0a31541d2858 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:18.514216 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbecf4-e847-41a5-aa82-0a31541d2858 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:18.603414 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9ef22-c917-4b09-8eb0-357cf3cadddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:18.603593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9ef22-c917-4b09-8eb0-357cf3cadddf admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:18.603936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9ef22-c917-4b09-8eb0-357cf3cadddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:18.604268 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9ef22-c917-4b09-8eb0-357cf3cadddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:18.605492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9ef22-c917-4b09-8eb0-357cf3cadddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:18.608123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9ef22-c917-4b09-8eb0-357cf3cadddf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:18.624841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489146fb-5585-4517-8c23-a9c4b5768627 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:18.626208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 893/1801] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:18.636297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489146fb-5585-4517-8c23-a9c4b5768627 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:18.642444 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457d4d0-5e67-4ce6-b1bd-a1997bf55b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:18.643441 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457d4d0-5e67-4ce6-b1bd-a1997bf55b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:18.644071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457d4d0-5e67-4ce6-b1bd-a1997bf55b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:18.644413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457d4d0-5e67-4ce6-b1bd-a1997bf55b0f admin admin] PATH_INFO: `/v3/projects/2d9a2d86417b4395ad94728e6e879ba8/users/3bba5acc965047579ed14b6700949f48/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:18.646465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457d4d0-5e67-4ce6-b1bd-a1997bf55b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d9a2d86417b4395ad94728e6e879ba8, user_id=3bba5acc965047579ed14b6700949f48, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:18.649600 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457d4d0-5e67-4ce6-b1bd-a1997bf55b0f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:18.663184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 909/1802] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:18 2022] PUT /identity/v3/projects/2d9a2d86417b4395ad94728e6e879ba8/users/3bba5acc965047579ed14b6700949f48/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:18.678736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152fcd7-fea0-41b6-97ea-ab1cb8773ebd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:18.679020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152fcd7-fea0-41b6-97ea-ab1cb8773ebd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:18.679316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152fcd7-fea0-41b6-97ea-ab1cb8773ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:18.709688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8152fcd7-fea0-41b6-97ea-ab1cb8773ebd None None] MFA Rules not processed for user `3bba5acc965047579ed14b6700949f48`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:18.711418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8152fcd7-fea0-41b6-97ea-ab1cb8773ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:18.797862 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 894/1803] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:18.810384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457d4d0-5e67-4ce6-b1bd-a1997bf55b0f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:18.812112 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5457d4d0-5e67-4ce6-b1bd-a1997bf55b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:18.869716 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457d4d0-5e67-4ce6-b1bd-a1997bf55b0f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:18.874013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd85706-c31b-41a3-8ac9-4c4f79540a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:18.874619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd85706-c31b-41a3-8ac9-4c4f79540a0a service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:18.874894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd85706-c31b-41a3-8ac9-4c4f79540a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:18.875196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd85706-c31b-41a3-8ac9-4c4f79540a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:18.881126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:18.885640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd85706-c31b-41a3-8ac9-4c4f79540a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:18.887631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd85706-c31b-41a3-8ac9-4c4f79540a0a service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:18.897964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 910/1804] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:19.445728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152fcd7-fea0-41b6-97ea-ab1cb8773ebd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:19.456407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152fcd7-fea0-41b6-97ea-ab1cb8773ebd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:19.459990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41e950-085e-412c-a0b4-b8c67dd7dbae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:19.460620 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41e950-085e-412c-a0b4-b8c67dd7dbae service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:19.460894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41e950-085e-412c-a0b4-b8c67dd7dbae service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:19.461162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41e950-085e-412c-a0b4-b8c67dd7dbae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:19.485500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:19.518349 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41e950-085e-412c-a0b4-b8c67dd7dbae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:19.519812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41e950-085e-412c-a0b4-b8c67dd7dbae service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:19.528910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 895/1805] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:25.363329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd85706-c31b-41a3-8ac9-4c4f79540a0a service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:25.365057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ffd85706-c31b-41a3-8ac9-4c4f79540a0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:25.566500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd85706-c31b-41a3-8ac9-4c4f79540a0a service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:25.582305 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:25.582970 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:25.583279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:25.583551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] PATH_INFO: `/v3/users/915dd32f0fc2401585361ae1285f5351` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:25.585111 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=915dd32f0fc2401585361ae1285f5351)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:25.589004 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:25.740195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '915dd32f0fc2401585361ae1285f5351'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:25.749310 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '915dd32f0fc2401585361ae1285f5351'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:25.749980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '915dd32f0fc2401585361ae1285f5351'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:25.786635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '915dd32f0fc2401585361ae1285f5351'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:25.798135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '915dd32f0fc2401585361ae1285f5351'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:25.807269 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 911/1806] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:25 2022] DELETE /identity/v3/users/915dd32f0fc2401585361ae1285f5351 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:25.812427 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a391d8-6eb5-402e-9a18-0856ddeb0fb0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:25.813632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a391d8-6eb5-402e-9a18-0856ddeb0fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:25.813972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a391d8-6eb5-402e-9a18-0856ddeb0fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:25.867209 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f0a391d8-6eb5-402e-9a18-0856ddeb0fb0 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:25.868430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f0a391d8-6eb5-402e-9a18-0856ddeb0fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:25.944812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 896/1807] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:25.954147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:25.955675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:26.025800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cc04a-7660-4d9b-88db-a0838c74db49 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:26.029807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aec3d5-04a7-452b-9f35-5f625a0c485e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:26.030421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aec3d5-04a7-452b-9f35-5f625a0c485e service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:26.030702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aec3d5-04a7-452b-9f35-5f625a0c485e service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:26.030973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aec3d5-04a7-452b-9f35-5f625a0c485e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:26.045400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aec3d5-04a7-452b-9f35-5f625a0c485e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:26.046890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aec3d5-04a7-452b-9f35-5f625a0c485e service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:26.056817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 912/1808] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:26.335427 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a391d8-6eb5-402e-9a18-0856ddeb0fb0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:26.336922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f0a391d8-6eb5-402e-9a18-0856ddeb0fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:26.350190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a391d8-6eb5-402e-9a18-0856ddeb0fb0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:26.354808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:26.355472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:26.355750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:26.356030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] PATH_INFO: `/v3/projects/e860c911879b418a8a28abbb67436755` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:26.358667 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e860c911879b418a8a28abbb67436755)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:26.360006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:26.409645 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e860c911879b418a8a28abbb67436755', 'request_id': 'req-c9d9a9dd-20c5-4f22-b3a3-992995b97635'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:26.415914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e860c911879b418a8a28abbb67436755', 'request_id': 'req-c9d9a9dd-20c5-4f22-b3a3-992995b97635'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:26.423398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e860c911879b418a8a28abbb67436755', 'request_id': 'req-c9d9a9dd-20c5-4f22-b3a3-992995b97635'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:26.436362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] The token cache is being invalidate because project e860c911879b418a8a28abbb67436755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:26.436662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:26.437450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:26.442483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 897/1809] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:26 2022] DELETE /identity/v3/projects/e860c911879b418a8a28abbb67436755 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:26.443731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aec3d5-04a7-452b-9f35-5f625a0c485e service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:26.453573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aec3d5-04a7-452b-9f35-5f625a0c485e service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:26.465307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:26.519441 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a98d9-f0a1-4bd8-a0e7-c551abfd9e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:26.520096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a98d9-f0a1-4bd8-a0e7-c551abfd9e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:26.520566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a98d9-f0a1-4bd8-a0e7-c551abfd9e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:26.520700 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a98d9-f0a1-4bd8-a0e7-c551abfd9e22 admin admin] PATH_INFO: `/v3/users/7e36d1c2cbe1451fb70072d312681bc1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:26.522366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a98d9-f0a1-4bd8-a0e7-c551abfd9e22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e36d1c2cbe1451fb70072d312681bc1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:26.523929 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a98d9-f0a1-4bd8-a0e7-c551abfd9e22 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:26.581136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-478a98d9-f0a1-4bd8-a0e7-c551abfd9e22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e36d1c2cbe1451fb70072d312681bc1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:26.588637 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-478a98d9-f0a1-4bd8-a0e7-c551abfd9e22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e36d1c2cbe1451fb70072d312681bc1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:26.589894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-478a98d9-f0a1-4bd8-a0e7-c551abfd9e22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e36d1c2cbe1451fb70072d312681bc1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:26.603634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-478a98d9-f0a1-4bd8-a0e7-c551abfd9e22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e36d1c2cbe1451fb70072d312681bc1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:26.617536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-478a98d9-f0a1-4bd8-a0e7-c551abfd9e22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e36d1c2cbe1451fb70072d312681bc1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:26.618794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 913/1810] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:26 2022] DELETE /identity/v3/users/7e36d1c2cbe1451fb70072d312681bc1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:26.926024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:26.927577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:26.959751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9a9dd-20c5-4f22-b3a3-992995b97635 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:27.025422 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:27.041565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94402a-071c-4a07-8721-0233cb832211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:27.042689 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94402a-071c-4a07-8721-0233cb832211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.042984 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94402a-071c-4a07-8721-0233cb832211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.043286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94402a-071c-4a07-8721-0233cb832211 admin admin] PATH_INFO: `/v3/projects/5f361393903c4472b8cb7909b1731392` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:27.044821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94402a-071c-4a07-8721-0233cb832211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f361393903c4472b8cb7909b1731392)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:27.046809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94402a-071c-4a07-8721-0233cb832211 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:27.098295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7f94402a-071c-4a07-8721-0233cb832211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f361393903c4472b8cb7909b1731392', 'request_id': 'req-7f94402a-071c-4a07-8721-0233cb832211'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:27.104259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7f94402a-071c-4a07-8721-0233cb832211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f361393903c4472b8cb7909b1731392', 'request_id': 'req-7f94402a-071c-4a07-8721-0233cb832211'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:27.111465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7f94402a-071c-4a07-8721-0233cb832211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f361393903c4472b8cb7909b1731392', 'request_id': 'req-7f94402a-071c-4a07-8721-0233cb832211'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:27.126644 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7f94402a-071c-4a07-8721-0233cb832211 admin admin] The token cache is being invalidate because project 5f361393903c4472b8cb7909b1731392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:27.126934 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7f94402a-071c-4a07-8721-0233cb832211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:27.127627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7f94402a-071c-4a07-8721-0233cb832211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:27.129510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 898/1811] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:26 2022] DELETE /identity/v3/projects/5f361393903c4472b8cb7909b1731392 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:27.134570 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dbb65-f72e-4be7-8c0f-cdc2dbce27f5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.134858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dbb65-f72e-4be7-8c0f-cdc2dbce27f5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.135160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dbb65-f72e-4be7-8c0f-cdc2dbce27f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:27.155534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6b5dbb65-f72e-4be7-8c0f-cdc2dbce27f5 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:27.156862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6b5dbb65-f72e-4be7-8c0f-cdc2dbce27f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:27.228897 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 914/1812] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:27.230661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed733d84-a853-4948-9a9f-4d1f058c44ab None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.231112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed733d84-a853-4948-9a9f-4d1f058c44ab None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.232084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed733d84-a853-4948-9a9f-4d1f058c44ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:27.254234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ed733d84-a853-4948-9a9f-4d1f058c44ab None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:27.255963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ed733d84-a853-4948-9a9f-4d1f058c44ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:27.266111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 899/1813] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:27.271325 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5dbb65-f72e-4be7-8c0f-cdc2dbce27f5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:27.287246 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5dbb65-f72e-4be7-8c0f-cdc2dbce27f5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:27.292204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5d056-b97d-4a74-b1c5-c5fd78fc7237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:27.292819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5d056-b97d-4a74-b1c5-c5fd78fc7237 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.293102 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5d056-b97d-4a74-b1c5-c5fd78fc7237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.293378 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5d056-b97d-4a74-b1c5-c5fd78fc7237 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:27.294539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5d056-b97d-4a74-b1c5-c5fd78fc7237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:15:27.294844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5d056-b97d-4a74-b1c5-c5fd78fc7237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:27.296548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5d056-b97d-4a74-b1c5-c5fd78fc7237 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:27.308705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 915/1814] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:15:27.313530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed733d84-a853-4948-9a9f-4d1f058c44ab None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:27.322840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed733d84-a853-4948-9a9f-4d1f058c44ab None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:27.328582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa1997-0d53-4545-a2d4-c86698f94549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:27.329190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa1997-0d53-4545-a2d4-c86698f94549 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.329539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa1997-0d53-4545-a2d4-c86698f94549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.329808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa1997-0d53-4545-a2d4-c86698f94549 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:27.330434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa1997-0d53-4545-a2d4-c86698f94549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:27.332524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa1997-0d53-4545-a2d4-c86698f94549 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:27.361100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 900/1815] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:27 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:27.365645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5d056-b97d-4a74-b1c5-c5fd78fc7237 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:27.376509 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5d056-b97d-4a74-b1c5-c5fd78fc7237 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:27.398829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:27.441910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5469eb-17ea-4a53-8a7d-9f186bf28789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:27.442524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5469eb-17ea-4a53-8a7d-9f186bf28789 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.442796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5469eb-17ea-4a53-8a7d-9f186bf28789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.443193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5469eb-17ea-4a53-8a7d-9f186bf28789 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:27.443866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5469eb-17ea-4a53-8a7d-9f186bf28789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:27.445796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5469eb-17ea-4a53-8a7d-9f186bf28789 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:27.471310 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 916/1816] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:27 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:27.475388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa1997-0d53-4545-a2d4-c86698f94549 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:27.486152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa1997-0d53-4545-a2d4-c86698f94549 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:27.490935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b4af6-ee63-48a3-a3ec-55e241e7bc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:27.491570 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b4af6-ee63-48a3-a3ec-55e241e7bc49 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.491853 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b4af6-ee63-48a3-a3ec-55e241e7bc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.492147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b4af6-ee63-48a3-a3ec-55e241e7bc49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:27.492778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b4af6-ee63-48a3-a3ec-55e241e7bc49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:27.494371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b4af6-ee63-48a3-a3ec-55e241e7bc49 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:27.505082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 901/1817] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:15:27.508985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5469eb-17ea-4a53-8a7d-9f186bf28789 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:27.519094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5469eb-17ea-4a53-8a7d-9f186bf28789 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:27.523591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c27b6d-1d0b-4683-90cd-86d3d207e3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:27.524187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c27b6d-1d0b-4683-90cd-86d3d207e3dc admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.524464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c27b6d-1d0b-4683-90cd-86d3d207e3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.524731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c27b6d-1d0b-4683-90cd-86d3d207e3dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:27.525362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c27b6d-1d0b-4683-90cd-86d3d207e3dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:27.526930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c27b6d-1d0b-4683-90cd-86d3d207e3dc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:27.537786 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 917/1818] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:27.541749 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b4af6-ee63-48a3-a3ec-55e241e7bc49 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:27.552032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b4af6-ee63-48a3-a3ec-55e241e7bc49 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:27.556746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96389065-5bd4-45ff-b668-f2ad398589b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:27.557395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96389065-5bd4-45ff-b668-f2ad398589b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.557673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96389065-5bd4-45ff-b668-f2ad398589b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.557948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96389065-5bd4-45ff-b668-f2ad398589b6 admin admin] PATH_INFO: `/v3/projects/8413fe47bd174c40a69b8361d88bbb03/users/5ca33fa3f727461a85b06a4e9878bdbf/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:27.578318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96389065-5bd4-45ff-b668-f2ad398589b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8413fe47bd174c40a69b8361d88bbb03, user_id=5ca33fa3f727461a85b06a4e9878bdbf, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:27.581166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96389065-5bd4-45ff-b668-f2ad398589b6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:27.596040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 902/1819] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:27 2022] PUT /identity/v3/projects/8413fe47bd174c40a69b8361d88bbb03/users/5ca33fa3f727461a85b06a4e9878bdbf/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:27.606964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c535f8-6e16-42a6-ae2e-60c22bad5662 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.607320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c535f8-6e16-42a6-ae2e-60c22bad5662 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.607585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c535f8-6e16-42a6-ae2e-60c22bad5662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:27.637035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-24c535f8-6e16-42a6-ae2e-60c22bad5662 None None] MFA Rules not processed for user `5ca33fa3f727461a85b06a4e9878bdbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:27.638416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-24c535f8-6e16-42a6-ae2e-60c22bad5662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:27.728728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 918/1820] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:27.730160 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96389065-5bd4-45ff-b668-f2ad398589b6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:27.800484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96389065-5bd4-45ff-b668-f2ad398589b6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:27.805200 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d49b8-dce5-4159-8c41-e5df98581d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:27.805896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d49b8-dce5-4159-8c41-e5df98581d73 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.806237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d49b8-dce5-4159-8c41-e5df98581d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.806632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d49b8-dce5-4159-8c41-e5df98581d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:27.807371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d49b8-dce5-4159-8c41-e5df98581d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:27.809308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d49b8-dce5-4159-8c41-e5df98581d73 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:27.830349 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:27.837973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 903/1821] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:27 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:27.842421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c535f8-6e16-42a6-ae2e-60c22bad5662 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:27.853297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c535f8-6e16-42a6-ae2e-60c22bad5662 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:27.911918 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338886f-c35c-4e13-b7b6-e9d3b8442751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:27.912529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338886f-c35c-4e13-b7b6-e9d3b8442751 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.912838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338886f-c35c-4e13-b7b6-e9d3b8442751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.913115 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338886f-c35c-4e13-b7b6-e9d3b8442751 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:27.913767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338886f-c35c-4e13-b7b6-e9d3b8442751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:27.915777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338886f-c35c-4e13-b7b6-e9d3b8442751 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:27.940533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 919/1822] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:27 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:27.944397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d49b8-dce5-4159-8c41-e5df98581d73 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:27.954922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d49b8-dce5-4159-8c41-e5df98581d73 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:27.959646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d8044-fe15-4b2a-8ba2-9e68a2ccd482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:27.960250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d8044-fe15-4b2a-8ba2-9e68a2ccd482 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.960531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d8044-fe15-4b2a-8ba2-9e68a2ccd482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.960803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d8044-fe15-4b2a-8ba2-9e68a2ccd482 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:27.961442 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d8044-fe15-4b2a-8ba2-9e68a2ccd482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:27.963033 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d8044-fe15-4b2a-8ba2-9e68a2ccd482 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:27.974725 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 904/1823] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:27.978839 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338886f-c35c-4e13-b7b6-e9d3b8442751 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:27.989851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338886f-c35c-4e13-b7b6-e9d3b8442751 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:27.994582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d8f05-2576-45da-aba2-65429ce19b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:27.995267 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d8f05-2576-45da-aba2-65429ce19b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:27.995550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d8f05-2576-45da-aba2-65429ce19b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:27.995825 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d8f05-2576-45da-aba2-65429ce19b7d admin admin] PATH_INFO: `/v3/projects/43f8940e0f90428f94be246b936c9fa7/users/6a25273cfdd1402da440b5a27d09d534/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:28.010633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d8f05-2576-45da-aba2-65429ce19b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=43f8940e0f90428f94be246b936c9fa7, user_id=6a25273cfdd1402da440b5a27d09d534, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:28.014030 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d8f05-2576-45da-aba2-65429ce19b7d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:28.017073 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:28.035467 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 920/1824] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:27 2022] PUT /identity/v3/projects/43f8940e0f90428f94be246b936c9fa7/users/6a25273cfdd1402da440b5a27d09d534/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:28.036662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d8044-fe15-4b2a-8ba2-9e68a2ccd482 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:28.049244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d8044-fe15-4b2a-8ba2-9e68a2ccd482 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:28.108974 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea8de1-c6a1-4595-af0a-ac22e86eafb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:28.109604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea8de1-c6a1-4595-af0a-ac22e86eafb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:28.109882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea8de1-c6a1-4595-af0a-ac22e86eafb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:28.110156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea8de1-c6a1-4595-af0a-ac22e86eafb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:28.110808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea8de1-c6a1-4595-af0a-ac22e86eafb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:28.112715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea8de1-c6a1-4595-af0a-ac22e86eafb0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:28.125481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 905/1825] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:28.126880 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d8f05-2576-45da-aba2-65429ce19b7d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:28.137182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d8f05-2576-45da-aba2-65429ce19b7d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:28.142599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d32e6a-6f3f-4224-94b8-2d4e832b9cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:28.143294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d32e6a-6f3f-4224-94b8-2d4e832b9cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:28.143571 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d32e6a-6f3f-4224-94b8-2d4e832b9cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:28.143842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d32e6a-6f3f-4224-94b8-2d4e832b9cd6 admin admin] PATH_INFO: `/v3/projects/43f8940e0f90428f94be246b936c9fa7/users/6a25273cfdd1402da440b5a27d09d534/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:28.145775 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d32e6a-6f3f-4224-94b8-2d4e832b9cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43f8940e0f90428f94be246b936c9fa7, user_id=6a25273cfdd1402da440b5a27d09d534, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:28.148471 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d32e6a-6f3f-4224-94b8-2d4e832b9cd6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:28.160302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 921/1826] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:28 2022] PUT /identity/v3/projects/43f8940e0f90428f94be246b936c9fa7/users/6a25273cfdd1402da440b5a27d09d534/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:28.171277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8b34-57cb-4e77-a3b1-cee4b5065481 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:28.171731 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8b34-57cb-4e77-a3b1-cee4b5065481 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:28.172139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8b34-57cb-4e77-a3b1-cee4b5065481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:28.206564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e25a8b34-57cb-4e77-a3b1-cee4b5065481 None None] MFA Rules not processed for user `6a25273cfdd1402da440b5a27d09d534`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:28.207811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e25a8b34-57cb-4e77-a3b1-cee4b5065481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:28.275656 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 906/1827] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:28.287227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d32e6a-6f3f-4224-94b8-2d4e832b9cd6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:28.289063 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d9d32e6a-6f3f-4224-94b8-2d4e832b9cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:28.348584 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d32e6a-6f3f-4224-94b8-2d4e832b9cd6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:28.352195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6fd6c-af10-42e1-9d09-48dd2abcd5ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:28.352833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6fd6c-af10-42e1-9d09-48dd2abcd5ff service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:28.353108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6fd6c-af10-42e1-9d09-48dd2abcd5ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:28.353377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6fd6c-af10-42e1-9d09-48dd2abcd5ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:28.362068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6fd6c-af10-42e1-9d09-48dd2abcd5ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:28.363776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6fd6c-af10-42e1-9d09-48dd2abcd5ff service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:28.373608 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 922/1828] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:29.066657 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a8b34-57cb-4e77-a3b1-cee4b5065481 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:29.233775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a8b34-57cb-4e77-a3b1-cee4b5065481 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:29.248627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:29.249301 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:29.249578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:29.249858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] PATH_INFO: `/v3/users/5ca33fa3f727461a85b06a4e9878bdbf` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:29.251030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca33fa3f727461a85b06a4e9878bdbf)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:29.252995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:29.282323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:29.372983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca33fa3f727461a85b06a4e9878bdbf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:29.402194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca33fa3f727461a85b06a4e9878bdbf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:29.430256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca33fa3f727461a85b06a4e9878bdbf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:29.431890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca33fa3f727461a85b06a4e9878bdbf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:29.450672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca33fa3f727461a85b06a4e9878bdbf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:29.474074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 907/1829] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:29 2022] DELETE /identity/v3/users/5ca33fa3f727461a85b06a4e9878bdbf => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:29.478677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42755ed9-5a68-4dea-86ab-19774420696e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:29.479222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42755ed9-5a68-4dea-86ab-19774420696e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:29.479644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42755ed9-5a68-4dea-86ab-19774420696e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:29.483187 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:29.485571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:29.516509 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-42755ed9-5a68-4dea-86ab-19774420696e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:29.517862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-42755ed9-5a68-4dea-86ab-19774420696e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:29.575942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd403a-bc3f-4215-8b03-531480a488db admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:29.583476 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39ef7c-200e-40d2-a4fc-ac427a0d82fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:29.584544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39ef7c-200e-40d2-a4fc-ac427a0d82fb service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:29.585254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39ef7c-200e-40d2-a4fc-ac427a0d82fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:29.585823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39ef7c-200e-40d2-a4fc-ac427a0d82fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:29.588779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6f39ef7c-200e-40d2-a4fc-ac427a0d82fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:29.650883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 923/1830] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:29.661078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42755ed9-5a68-4dea-86ab-19774420696e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:29.662485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-42755ed9-5a68-4dea-86ab-19774420696e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:29.684693 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f39ef7c-200e-40d2-a4fc-ac427a0d82fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:29.686979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f39ef7c-200e-40d2-a4fc-ac427a0d82fb service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:29.698400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 908/1831] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:29 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:15:29.765294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42755ed9-5a68-4dea-86ab-19774420696e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:29.770464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdb852-8a1a-4c61-bfee-dfdae671db5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:29.771334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdb852-8a1a-4c61-bfee-dfdae671db5e service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:29.771769 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdb852-8a1a-4c61-bfee-dfdae671db5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:29.772193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdb852-8a1a-4c61-bfee-dfdae671db5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:29.788450 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdb852-8a1a-4c61-bfee-dfdae671db5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:29.790775 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdb852-8a1a-4c61-bfee-dfdae671db5e service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:29.809149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 924/1832] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:15:30.613203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39ef7c-200e-40d2-a4fc-ac427a0d82fb service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:30.615239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6f39ef7c-200e-40d2-a4fc-ac427a0d82fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:30.636686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39ef7c-200e-40d2-a4fc-ac427a0d82fb service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:30.642218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3e5fe-8e96-473a-8156-14cfc36bc730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:30.643322 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e5fe-8e96-473a-8156-14cfc36bc730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:30.643322 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e5fe-8e96-473a-8156-14cfc36bc730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:30.644002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e5fe-8e96-473a-8156-14cfc36bc730 admin admin] PATH_INFO: `/v3/projects/8413fe47bd174c40a69b8361d88bbb03` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:30.645797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3e5fe-8e96-473a-8156-14cfc36bc730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8413fe47bd174c40a69b8361d88bbb03)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:30.647915 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3e5fe-8e96-473a-8156-14cfc36bc730 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:30.711034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:30.721879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3cb3e5fe-8e96-473a-8156-14cfc36bc730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8413fe47bd174c40a69b8361d88bbb03', 'request_id': 'req-3cb3e5fe-8e96-473a-8156-14cfc36bc730'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:30.728095 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3cb3e5fe-8e96-473a-8156-14cfc36bc730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8413fe47bd174c40a69b8361d88bbb03', 'request_id': 'req-3cb3e5fe-8e96-473a-8156-14cfc36bc730'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:30.737937 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3cb3e5fe-8e96-473a-8156-14cfc36bc730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8413fe47bd174c40a69b8361d88bbb03', 'request_id': 'req-3cb3e5fe-8e96-473a-8156-14cfc36bc730'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:30.754678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3cb3e5fe-8e96-473a-8156-14cfc36bc730 admin admin] The token cache is being invalidate because project 8413fe47bd174c40a69b8361d88bbb03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:30.754958 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3cb3e5fe-8e96-473a-8156-14cfc36bc730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:30.755716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3cb3e5fe-8e96-473a-8156-14cfc36bc730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:30.757514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 909/1833] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:30 2022] DELETE /identity/v3/projects/8413fe47bd174c40a69b8361d88bbb03 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:30.762610 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdb852-8a1a-4c61-bfee-dfdae671db5e service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:30.775915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdb852-8a1a-4c61-bfee-dfdae671db5e service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:30.810988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:30.850189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:30.851255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:30.852070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:30.852403 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] PATH_INFO: `/v3/users/6a25273cfdd1402da440b5a27d09d534` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:30.853791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a25273cfdd1402da440b5a27d09d534)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:30.856001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:30.933836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a25273cfdd1402da440b5a27d09d534'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:30.948551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a25273cfdd1402da440b5a27d09d534'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:30.949435 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a25273cfdd1402da440b5a27d09d534'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:30.968487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a25273cfdd1402da440b5a27d09d534'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:30.976722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a25273cfdd1402da440b5a27d09d534'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:30.988666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 925/1834] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:30 2022] DELETE /identity/v3/users/6a25273cfdd1402da440b5a27d09d534 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:31.368695 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff0fbb-2811-4486-8f69-6a53ae6db170 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:31.368982 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff0fbb-2811-4486-8f69-6a53ae6db170 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:31.369261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff0fbb-2811-4486-8f69-6a53ae6db170 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:31.370639 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 910/1835] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:15:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:15:31.390457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:31.392412 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:31.396284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2510c-11d8-482b-8fb6-bd3f03228a65 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:31.396615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2510c-11d8-482b-8fb6-bd3f03228a65 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:31.396894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2510c-11d8-482b-8fb6-bd3f03228a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:31.444642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-09a2510c-11d8-482b-8fb6-bd3f03228a65 None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:31.446828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-09a2510c-11d8-482b-8fb6-bd3f03228a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:31.576350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8175b-9bf0-4569-b674-007d5e825768 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:31.597095 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd0a2e-590e-490c-8156-23bae4b8f325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:31.597095 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd0a2e-590e-490c-8156-23bae4b8f325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:31.597095 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd0a2e-590e-490c-8156-23bae4b8f325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:31.597095 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd0a2e-590e-490c-8156-23bae4b8f325 admin admin] PATH_INFO: `/v3/projects/43f8940e0f90428f94be246b936c9fa7` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:31.600387 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd0a2e-590e-490c-8156-23bae4b8f325 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43f8940e0f90428f94be246b936c9fa7)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:31.602694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd0a2e-590e-490c-8156-23bae4b8f325 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:31.644292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 911/1836] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:15:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:31.723721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27cd0a2e-590e-490c-8156-23bae4b8f325 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43f8940e0f90428f94be246b936c9fa7', 'request_id': 'req-27cd0a2e-590e-490c-8156-23bae4b8f325'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:31.726688 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:31.739665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27cd0a2e-590e-490c-8156-23bae4b8f325 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43f8940e0f90428f94be246b936c9fa7', 'request_id': 'req-27cd0a2e-590e-490c-8156-23bae4b8f325'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:31.749786 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27cd0a2e-590e-490c-8156-23bae4b8f325 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43f8940e0f90428f94be246b936c9fa7', 'request_id': 'req-27cd0a2e-590e-490c-8156-23bae4b8f325'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:31.768771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2510c-11d8-482b-8fb6-bd3f03228a65 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:31.770528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-09a2510c-11d8-482b-8fb6-bd3f03228a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:31.774977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27cd0a2e-590e-490c-8156-23bae4b8f325 admin admin] The token cache is being invalidate because project 43f8940e0f90428f94be246b936c9fa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:31.775511 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27cd0a2e-590e-490c-8156-23bae4b8f325 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:31.777485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27cd0a2e-590e-490c-8156-23bae4b8f325 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:31.782808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 926/1837] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:31 2022] DELETE /identity/v3/projects/43f8940e0f90428f94be246b936c9fa7 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:31.784803 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acabaf-641c-4141-8664-bce991fc6979 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:31.785096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acabaf-641c-4141-8664-bce991fc6979 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:31.785375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acabaf-641c-4141-8664-bce991fc6979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:31.800545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2510c-11d8-482b-8fb6-bd3f03228a65 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:31.826046 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b7acabaf-641c-4141-8664-bce991fc6979 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:31.827326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b7acabaf-641c-4141-8664-bce991fc6979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:31.945204 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ec586-ade9-4c03-b320-77a078f470fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:31.947772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ec586-ade9-4c03-b320-77a078f470fa service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:31.948685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ec586-ade9-4c03-b320-77a078f470fa service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:31.948980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ec586-ade9-4c03-b320-77a078f470fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:31.949280 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 927/1838] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:31.959223 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d55ec586-ade9-4c03-b320-77a078f470fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:31.959762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b275e51-645f-406f-8a24-b0c2756b92ba None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:31.960153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b275e51-645f-406f-8a24-b0c2756b92ba None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:31.960523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b275e51-645f-406f-8a24-b0c2756b92ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:31.998260 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ec586-ade9-4c03-b320-77a078f470fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:31.999921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ec586-ade9-4c03-b320-77a078f470fa service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:32.002340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3b275e51-645f-406f-8a24-b0c2756b92ba None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:32.004445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3b275e51-645f-406f-8a24-b0c2756b92ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:32.024878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 928/1839] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:32.026452 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b275e51-645f-406f-8a24-b0c2756b92ba None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:32.030613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 912/1840] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:15:32.042209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b275e51-645f-406f-8a24-b0c2756b92ba None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:32.050211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b796a2-e77d-49c9-840b-aafb38a7c941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:32.050851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b796a2-e77d-49c9-840b-aafb38a7c941 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.051816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b796a2-e77d-49c9-840b-aafb38a7c941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.052106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b796a2-e77d-49c9-840b-aafb38a7c941 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.053466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b796a2-e77d-49c9-840b-aafb38a7c941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:15:32.053882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b796a2-e77d-49c9-840b-aafb38a7c941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:32.056548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b796a2-e77d-49c9-840b-aafb38a7c941 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:32.079897 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 929/1841] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:15:32.081708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ec586-ade9-4c03-b320-77a078f470fa service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:32.103229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ec586-ade9-4c03-b320-77a078f470fa service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:32.110075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75995e35-505c-4c3c-a2ae-25c769b12ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:32.110890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75995e35-505c-4c3c-a2ae-25c769b12ede admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.111238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75995e35-505c-4c3c-a2ae-25c769b12ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.111514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75995e35-505c-4c3c-a2ae-25c769b12ede admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.112144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75995e35-505c-4c3c-a2ae-25c769b12ede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:32.114449 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75995e35-505c-4c3c-a2ae-25c769b12ede admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:32.156595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 913/1842] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:32 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:32.161180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b796a2-e77d-49c9-840b-aafb38a7c941 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:32.172738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b796a2-e77d-49c9-840b-aafb38a7c941 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:32.252800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860173f-bfe5-4632-863c-227d96479574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:32.253441 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860173f-bfe5-4632-863c-227d96479574 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.253805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860173f-bfe5-4632-863c-227d96479574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.254033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860173f-bfe5-4632-863c-227d96479574 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.254610 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860173f-bfe5-4632-863c-227d96479574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:32.256579 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860173f-bfe5-4632-863c-227d96479574 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:32.288475 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 930/1843] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:32 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:32.308160 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75995e35-505c-4c3c-a2ae-25c769b12ede admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:32.321185 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75995e35-505c-4c3c-a2ae-25c769b12ede admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:32.326269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494c2eb-73d7-40c7-9984-547d1ddc747e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:32.326908 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494c2eb-73d7-40c7-9984-547d1ddc747e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.327228 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494c2eb-73d7-40c7-9984-547d1ddc747e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.327505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494c2eb-73d7-40c7-9984-547d1ddc747e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.328213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494c2eb-73d7-40c7-9984-547d1ddc747e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:32.329827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494c2eb-73d7-40c7-9984-547d1ddc747e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:32.340107 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 914/1844] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:32.346462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860173f-bfe5-4632-863c-227d96479574 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:32.357090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860173f-bfe5-4632-863c-227d96479574 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:32.362679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73be6e6-f906-4d32-9694-2bf397b13d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:32.363337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73be6e6-f906-4d32-9694-2bf397b13d96 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.363617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73be6e6-f906-4d32-9694-2bf397b13d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.363888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73be6e6-f906-4d32-9694-2bf397b13d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.364539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73be6e6-f906-4d32-9694-2bf397b13d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:32.366143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73be6e6-f906-4d32-9694-2bf397b13d96 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:32.381723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 931/1845] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:32.383524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494c2eb-73d7-40c7-9984-547d1ddc747e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:32.393890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494c2eb-73d7-40c7-9984-547d1ddc747e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:32.398914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb095dc-aecf-4161-834b-2c6de3ec6d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:32.399600 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb095dc-aecf-4161-834b-2c6de3ec6d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.399878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb095dc-aecf-4161-834b-2c6de3ec6d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.400149 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb095dc-aecf-4161-834b-2c6de3ec6d86 admin admin] PATH_INFO: `/v3/projects/6509bc9106d04912a51222396bd74741/users/a251d538d8d949e080e691a11929b57b/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.409514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:32.416500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb095dc-aecf-4161-834b-2c6de3ec6d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6509bc9106d04912a51222396bd74741, user_id=a251d538d8d949e080e691a11929b57b, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:32.420086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb095dc-aecf-4161-834b-2c6de3ec6d86 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:32.435726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 915/1846] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:32 2022] PUT /identity/v3/projects/6509bc9106d04912a51222396bd74741/users/a251d538d8d949e080e691a11929b57b/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:32.446647 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91063a05-0711-4b5b-895d-3382d0343b23 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.447433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91063a05-0711-4b5b-895d-3382d0343b23 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.447433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91063a05-0711-4b5b-895d-3382d0343b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.472312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb095dc-aecf-4161-834b-2c6de3ec6d86 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:32.474084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4fb095dc-aecf-4161-834b-2c6de3ec6d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:32.500195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-91063a05-0711-4b5b-895d-3382d0343b23 None None] MFA Rules not processed for user `a251d538d8d949e080e691a11929b57b`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:32.504950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-91063a05-0711-4b5b-895d-3382d0343b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:32.546411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:32.558549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb095dc-aecf-4161-834b-2c6de3ec6d86 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:32.564328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f633636-edf0-4fd8-8321-6abbb8f76851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:32.565068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f633636-edf0-4fd8-8321-6abbb8f76851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.565424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f633636-edf0-4fd8-8321-6abbb8f76851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.565700 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f633636-edf0-4fd8-8321-6abbb8f76851 admin admin] PATH_INFO: `/v3/users/1505cde569dc4edfb7c6ff8ae34bcd52` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.566846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f633636-edf0-4fd8-8321-6abbb8f76851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1505cde569dc4edfb7c6ff8ae34bcd52)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:32.568892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f633636-edf0-4fd8-8321-6abbb8f76851 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:32.590123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 932/1847] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:32.595283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91063a05-0711-4b5b-895d-3382d0343b23 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:32.608801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91063a05-0711-4b5b-895d-3382d0343b23 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:32.614372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441f9be-1d25-4ffd-8476-a3aac751ff63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:32.615075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441f9be-1d25-4ffd-8476-a3aac751ff63 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.615767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441f9be-1d25-4ffd-8476-a3aac751ff63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.615767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441f9be-1d25-4ffd-8476-a3aac751ff63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.616461 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441f9be-1d25-4ffd-8476-a3aac751ff63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:32.618768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441f9be-1d25-4ffd-8476-a3aac751ff63 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:32.633804 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5f633636-edf0-4fd8-8321-6abbb8f76851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1505cde569dc4edfb7c6ff8ae34bcd52'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:32.646846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5f633636-edf0-4fd8-8321-6abbb8f76851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1505cde569dc4edfb7c6ff8ae34bcd52'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:32.655083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 933/1848] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:32 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:32.657767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441f9be-1d25-4ffd-8476-a3aac751ff63 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:32.662376 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5f633636-edf0-4fd8-8321-6abbb8f76851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1505cde569dc4edfb7c6ff8ae34bcd52'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:32.663092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5f633636-edf0-4fd8-8321-6abbb8f76851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1505cde569dc4edfb7c6ff8ae34bcd52'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:32.671004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5f633636-edf0-4fd8-8321-6abbb8f76851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1505cde569dc4edfb7c6ff8ae34bcd52'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:32.677020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 916/1849] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:32 2022] DELETE /identity/v3/users/1505cde569dc4edfb7c6ff8ae34bcd52 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:32.684429 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb80fd-92f0-49d2-8cbf-6b4e8d068376 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.684725 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb80fd-92f0-49d2-8cbf-6b4e8d068376 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.685226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb80fd-92f0-49d2-8cbf-6b4e8d068376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.705775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-64cb80fd-92f0-49d2-8cbf-6b4e8d068376 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:32.707029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-64cb80fd-92f0-49d2-8cbf-6b4e8d068376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:32.721347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441f9be-1d25-4ffd-8476-a3aac751ff63 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:32.774098 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 917/1850] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:32.783409 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8987716-218d-47aa-8693-d2cf4ff5a9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:32.784031 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8987716-218d-47aa-8693-d2cf4ff5a9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.784314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8987716-218d-47aa-8693-d2cf4ff5a9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.784586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8987716-218d-47aa-8693-d2cf4ff5a9b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.785222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8987716-218d-47aa-8693-d2cf4ff5a9b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:32.786234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb80fd-92f0-49d2-8cbf-6b4e8d068376 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:32.787236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8987716-218d-47aa-8693-d2cf4ff5a9b4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:32.788175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-64cb80fd-92f0-49d2-8cbf-6b4e8d068376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:32.799092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:32.814658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 934/1851] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:32 2022] POST /identity/v3/users => generated 298 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:32.815796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8987716-218d-47aa-8693-d2cf4ff5a9b4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:32.817876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b8987716-218d-47aa-8693-d2cf4ff5a9b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:32.831083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8987716-218d-47aa-8693-d2cf4ff5a9b4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:32.836588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de6825-d3d6-4ba0-abc8-268eae0381ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:32.837408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de6825-d3d6-4ba0-abc8-268eae0381ab admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.837756 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de6825-d3d6-4ba0-abc8-268eae0381ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.838102 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de6825-d3d6-4ba0-abc8-268eae0381ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.838828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de6825-d3d6-4ba0-abc8-268eae0381ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:32.840589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de6825-d3d6-4ba0-abc8-268eae0381ab admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:32.853176 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 935/1852] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:32.858521 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de6825-d3d6-4ba0-abc8-268eae0381ab admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:32.861739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb80fd-92f0-49d2-8cbf-6b4e8d068376 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:32.865777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65ea78-1970-417f-9829-7494ecc8bafd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:32.866401 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65ea78-1970-417f-9829-7494ecc8bafd service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.866678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65ea78-1970-417f-9829-7494ecc8bafd service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.866955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65ea78-1970-417f-9829-7494ecc8bafd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.878127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de6825-d3d6-4ba0-abc8-268eae0381ab admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:32.879166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65ea78-1970-417f-9829-7494ecc8bafd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:32.880169 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65ea78-1970-417f-9829-7494ecc8bafd service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:32.883673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2be65-d6c5-4e82-9e90-79e1abe862b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:32.884368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2be65-d6c5-4e82-9e90-79e1abe862b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:32.884633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2be65-d6c5-4e82-9e90-79e1abe862b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:32.885124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2be65-d6c5-4e82-9e90-79e1abe862b5 admin admin] PATH_INFO: `/v3/projects/2a060cc764074eff839aa7c74335b94a/users/dc3049ba43e34449a1ae9f00612181fe/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:32.892250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 918/1853] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:15:32.905009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2be65-d6c5-4e82-9e90-79e1abe862b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a060cc764074eff839aa7c74335b94a, user_id=dc3049ba43e34449a1ae9f00612181fe, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:32.910189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2be65-d6c5-4e82-9e90-79e1abe862b5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:32.926814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 936/1854] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:32 2022] PUT /identity/v3/projects/2a060cc764074eff839aa7c74335b94a/users/dc3049ba43e34449a1ae9f00612181fe/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:32.930995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65ea78-1970-417f-9829-7494ecc8bafd service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:32.941801 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65ea78-1970-417f-9829-7494ecc8bafd service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:33.042357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d03839-1f44-4467-b1be-586799a7e9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:33.043163 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d03839-1f44-4467-b1be-586799a7e9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:33.043340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d03839-1f44-4467-b1be-586799a7e9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:33.043633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d03839-1f44-4467-b1be-586799a7e9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:33.045054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d03839-1f44-4467-b1be-586799a7e9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:33.047078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d03839-1f44-4467-b1be-586799a7e9d4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:33.070860 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 919/1855] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:33.076138 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2be65-d6c5-4e82-9e90-79e1abe862b5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:33.092653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2be65-d6c5-4e82-9e90-79e1abe862b5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:33.099860 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3217d6-593b-4e0e-afbe-b80e22191037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:33.101333 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3217d6-593b-4e0e-afbe-b80e22191037 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:33.101810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3217d6-593b-4e0e-afbe-b80e22191037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:33.102378 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3217d6-593b-4e0e-afbe-b80e22191037 admin admin] PATH_INFO: `/v3/projects/2a060cc764074eff839aa7c74335b94a/users/dc3049ba43e34449a1ae9f00612181fe/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:33.104932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3217d6-593b-4e0e-afbe-b80e22191037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a060cc764074eff839aa7c74335b94a, user_id=dc3049ba43e34449a1ae9f00612181fe, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:33.110622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3217d6-593b-4e0e-afbe-b80e22191037 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:33.116616 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:33.133809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 937/1856] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:33 2022] PUT /identity/v3/projects/2a060cc764074eff839aa7c74335b94a/users/dc3049ba43e34449a1ae9f00612181fe/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:33.147903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabef36b-e48c-4e2e-add1-494f806470a8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:33.148182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabef36b-e48c-4e2e-add1-494f806470a8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:33.148678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabef36b-e48c-4e2e-add1-494f806470a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:33.181328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-eabef36b-e48c-4e2e-add1-494f806470a8 None None] MFA Rules not processed for user `dc3049ba43e34449a1ae9f00612181fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:33.182496 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eabef36b-e48c-4e2e-add1-494f806470a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:33.244573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3217d6-593b-4e0e-afbe-b80e22191037 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:33.246466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3a3217d6-593b-4e0e-afbe-b80e22191037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:33.287121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 920/1857] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:33.296735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabef36b-e48c-4e2e-add1-494f806470a8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:33.298168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eabef36b-e48c-4e2e-add1-494f806470a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:33.339799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3217d6-593b-4e0e-afbe-b80e22191037 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:33.345168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:33.345701 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:33.346806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:33.347328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:33.347753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] PATH_INFO: `/v3/projects/69948db1d7a546708fb9f07b9e047722` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:33.350169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=69948db1d7a546708fb9f07b9e047722)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:33.352296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:33.392696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabef36b-e48c-4e2e-add1-494f806470a8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:33.397309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b0bbe-4d46-4608-bf2a-e9d3cf6d8ec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:33.398032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b0bbe-4d46-4608-bf2a-e9d3cf6d8ec5 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:33.398544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b0bbe-4d46-4608-bf2a-e9d3cf6d8ec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:33.398864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b0bbe-4d46-4608-bf2a-e9d3cf6d8ec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:33.400876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b24b0bbe-4d46-4608-bf2a-e9d3cf6d8ec5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:33.426230 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69948db1d7a546708fb9f07b9e047722', 'request_id': 'req-d4144a48-161a-4060-b7a9-953d5b9c17ec'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:33.446782 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69948db1d7a546708fb9f07b9e047722', 'request_id': 'req-d4144a48-161a-4060-b7a9-953d5b9c17ec'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:33.453950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69948db1d7a546708fb9f07b9e047722', 'request_id': 'req-d4144a48-161a-4060-b7a9-953d5b9c17ec'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:33.470104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] The token cache is being invalidate because project 69948db1d7a546708fb9f07b9e047722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:33.470487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:33.471303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:33.473118 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 938/1858] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:33 2022] DELETE /identity/v3/projects/69948db1d7a546708fb9f07b9e047722 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:33.479888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:33.482799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:33.492313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b0bbe-4d46-4608-bf2a-e9d3cf6d8ec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:33.494407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b0bbe-4d46-4608-bf2a-e9d3cf6d8ec5 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:33.508792 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 921/1859] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:15:33.657011 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4144a48-161a-4060-b7a9-953d5b9c17ec admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:33.662713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:33.663261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:33.663624 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:33.663971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] PATH_INFO: `/v3/users/8ecb4007b2024b31b7d6f04c16ccf66a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:33.665204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ecb4007b2024b31b7d6f04c16ccf66a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:33.667264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:33.700557 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=11 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:33.728527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ecb4007b2024b31b7d6f04c16ccf66a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:33.740838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ecb4007b2024b31b7d6f04c16ccf66a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:33.741568 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ecb4007b2024b31b7d6f04c16ccf66a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:33.756823 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ecb4007b2024b31b7d6f04c16ccf66a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:33.765742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ecb4007b2024b31b7d6f04c16ccf66a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:33.772712 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 939/1860] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:33 2022] DELETE /identity/v3/users/8ecb4007b2024b31b7d6f04c16ccf66a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:34.194068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b0bbe-4d46-4608-bf2a-e9d3cf6d8ec5 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:34.206523 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b0bbe-4d46-4608-bf2a-e9d3cf6d8ec5 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:34.288063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:34.288696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:34.289006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:34.289439 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] PATH_INFO: `/v3/projects/f2d566baeace4244869c3c7a134188ca` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:34.290587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2d566baeace4244869c3c7a134188ca)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:34.292591 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:34.345826 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2d566baeace4244869c3c7a134188ca', 'request_id': 'req-f4a4d648-0221-4758-8038-3a1b3c627f11'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:34.352116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2d566baeace4244869c3c7a134188ca', 'request_id': 'req-f4a4d648-0221-4758-8038-3a1b3c627f11'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:34.358835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2d566baeace4244869c3c7a134188ca', 'request_id': 'req-f4a4d648-0221-4758-8038-3a1b3c627f11'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:34.373228 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] The token cache is being invalidate because project f2d566baeace4244869c3c7a134188ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:34.373517 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:34.374164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:34.375703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 922/1861] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:34 2022] DELETE /identity/v3/projects/f2d566baeace4244869c3c7a134188ca => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:34.379834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:34.381370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:34.396114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4ac18-a984-4abb-8465-5f9ca92dbc34 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:34.458537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c61e88-38c9-4372-b182-03dd3c10575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:34.459174 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c61e88-38c9-4372-b182-03dd3c10575e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:34.459508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c61e88-38c9-4372-b182-03dd3c10575e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:34.459783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c61e88-38c9-4372-b182-03dd3c10575e admin admin] PATH_INFO: `/v3/users/3bba5acc965047579ed14b6700949f48` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:34.462312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c61e88-38c9-4372-b182-03dd3c10575e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bba5acc965047579ed14b6700949f48)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:34.462869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c61e88-38c9-4372-b182-03dd3c10575e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:34.512104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e1c61e88-38c9-4372-b182-03dd3c10575e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bba5acc965047579ed14b6700949f48'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:34.519392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e1c61e88-38c9-4372-b182-03dd3c10575e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bba5acc965047579ed14b6700949f48'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:34.520045 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e1c61e88-38c9-4372-b182-03dd3c10575e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bba5acc965047579ed14b6700949f48'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:34.532910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e1c61e88-38c9-4372-b182-03dd3c10575e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bba5acc965047579ed14b6700949f48'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:34.539982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e1c61e88-38c9-4372-b182-03dd3c10575e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bba5acc965047579ed14b6700949f48'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:34.546312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 940/1862] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:34 2022] DELETE /identity/v3/users/3bba5acc965047579ed14b6700949f48 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:34.980451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:34.982521 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:34.998296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4d648-0221-4758-8038-3a1b3c627f11 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:35.017417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:35.068715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e96bb4-b919-4756-8633-1c4882ac8c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:35.069487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e96bb4-b919-4756-8633-1c4882ac8c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.069763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e96bb4-b919-4756-8633-1c4882ac8c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.070033 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e96bb4-b919-4756-8633-1c4882ac8c89 admin admin] PATH_INFO: `/v3/projects/2d9a2d86417b4395ad94728e6e879ba8` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.071253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e96bb4-b919-4756-8633-1c4882ac8c89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d9a2d86417b4395ad94728e6e879ba8)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:35.073208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e96bb4-b919-4756-8633-1c4882ac8c89 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:35.119344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-06e96bb4-b919-4756-8633-1c4882ac8c89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d9a2d86417b4395ad94728e6e879ba8', 'request_id': 'req-06e96bb4-b919-4756-8633-1c4882ac8c89'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:35.125450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-06e96bb4-b919-4756-8633-1c4882ac8c89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d9a2d86417b4395ad94728e6e879ba8', 'request_id': 'req-06e96bb4-b919-4756-8633-1c4882ac8c89'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:35.131590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-06e96bb4-b919-4756-8633-1c4882ac8c89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d9a2d86417b4395ad94728e6e879ba8', 'request_id': 'req-06e96bb4-b919-4756-8633-1c4882ac8c89'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:35.143665 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-06e96bb4-b919-4756-8633-1c4882ac8c89 admin admin] The token cache is being invalidate because project 2d9a2d86417b4395ad94728e6e879ba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:35.143954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-06e96bb4-b919-4756-8633-1c4882ac8c89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:35.144576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-06e96bb4-b919-4756-8633-1c4882ac8c89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:35.146081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 923/1863] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:34 2022] DELETE /identity/v3/projects/2d9a2d86417b4395ad94728e6e879ba8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:35.151910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae7fe5-3188-4470-8e63-c4062faa43cf None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.152222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae7fe5-3188-4470-8e63-c4062faa43cf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.152499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae7fe5-3188-4470-8e63-c4062faa43cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.173696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-00ae7fe5-3188-4470-8e63-c4062faa43cf None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:35.174902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-00ae7fe5-3188-4470-8e63-c4062faa43cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:35.192688 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:35.237996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 941/1864] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:35.247870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3b215-87cd-4d59-9ff3-e1e5841fa352 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.248476 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3b215-87cd-4d59-9ff3-e1e5841fa352 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.248476 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3b215-87cd-4d59-9ff3-e1e5841fa352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.273299 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-78b3b215-87cd-4d59-9ff3-e1e5841fa352 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:35.274524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-78b3b215-87cd-4d59-9ff3-e1e5841fa352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:35.282567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 924/1865] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:35.286947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae7fe5-3188-4470-8e63-c4062faa43cf None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:35.297038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae7fe5-3188-4470-8e63-c4062faa43cf None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:35.302011 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a009b-5821-426a-90fd-bdcbcc33dfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:35.302833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a009b-5821-426a-90fd-bdcbcc33dfb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.303155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a009b-5821-426a-90fd-bdcbcc33dfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.303437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a009b-5821-426a-90fd-bdcbcc33dfb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.304124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a009b-5821-426a-90fd-bdcbcc33dfb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:15:35.304428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a009b-5821-426a-90fd-bdcbcc33dfb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:35.306063 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a009b-5821-426a-90fd-bdcbcc33dfb4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:35.323566 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3b215-87cd-4d59-9ff3-e1e5841fa352 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:35.326306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 942/1866] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:15:35.334038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3b215-87cd-4d59-9ff3-e1e5841fa352 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:35.338756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3f269-5fa6-4be0-ad3f-98c8dd5639c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:35.339400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f269-5fa6-4be0-ad3f-98c8dd5639c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.339685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f269-5fa6-4be0-ad3f-98c8dd5639c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.339962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f269-5fa6-4be0-ad3f-98c8dd5639c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.340659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3f269-5fa6-4be0-ad3f-98c8dd5639c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:35.342578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3f269-5fa6-4be0-ad3f-98c8dd5639c2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:35.374962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 925/1867] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:35 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:35.376100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a009b-5821-426a-90fd-bdcbcc33dfb4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:35.386128 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a009b-5821-426a-90fd-bdcbcc33dfb4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:35.448980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa6e38-ccf7-4efc-9c37-968b4da19be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:35.449597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa6e38-ccf7-4efc-9c37-968b4da19be6 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.449959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa6e38-ccf7-4efc-9c37-968b4da19be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.450453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa6e38-ccf7-4efc-9c37-968b4da19be6 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.451182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa6e38-ccf7-4efc-9c37-968b4da19be6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:35.453092 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa6e38-ccf7-4efc-9c37-968b4da19be6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:35.475301 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 943/1868] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:35 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:35.479422 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3f269-5fa6-4be0-ad3f-98c8dd5639c2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:35.490684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3f269-5fa6-4be0-ad3f-98c8dd5639c2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:35.495280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a3c46-39ce-44ef-9467-ef98c5cfeb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:35.495918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a3c46-39ce-44ef-9467-ef98c5cfeb58 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.496194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a3c46-39ce-44ef-9467-ef98c5cfeb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.496465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a3c46-39ce-44ef-9467-ef98c5cfeb58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.497253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a3c46-39ce-44ef-9467-ef98c5cfeb58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:35.498854 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a3c46-39ce-44ef-9467-ef98c5cfeb58 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:35.509237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 926/1869] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:35.513900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa6e38-ccf7-4efc-9c37-968b4da19be6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:35.523867 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa6e38-ccf7-4efc-9c37-968b4da19be6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:35.529000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556451be-544f-460d-bd25-3ccb24461f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:35.529609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556451be-544f-460d-bd25-3ccb24461f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.529887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556451be-544f-460d-bd25-3ccb24461f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.530188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556451be-544f-460d-bd25-3ccb24461f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.530857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556451be-544f-460d-bd25-3ccb24461f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:35.532495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556451be-544f-460d-bd25-3ccb24461f01 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:35.544220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 944/1870] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:35.548747 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a3c46-39ce-44ef-9467-ef98c5cfeb58 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:35.558603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a3c46-39ce-44ef-9467-ef98c5cfeb58 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:35.563452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc59382-ea4e-44d2-834d-ef9566ba7437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:35.564129 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc59382-ea4e-44d2-834d-ef9566ba7437 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.564403 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc59382-ea4e-44d2-834d-ef9566ba7437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.564663 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc59382-ea4e-44d2-834d-ef9566ba7437 admin admin] PATH_INFO: `/v3/projects/30e7d8b7dfec48f5b273781e3f4ccdc4/users/ba49c1c485d64f4f8cdf81d036cf405b/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.581878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc59382-ea4e-44d2-834d-ef9566ba7437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30e7d8b7dfec48f5b273781e3f4ccdc4, user_id=ba49c1c485d64f4f8cdf81d036cf405b, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:35.584571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc59382-ea4e-44d2-834d-ef9566ba7437 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:35.601029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 927/1871] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:35 2022] PUT /identity/v3/projects/30e7d8b7dfec48f5b273781e3f4ccdc4/users/ba49c1c485d64f4f8cdf81d036cf405b/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:35.613132 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe573721-a03b-4ee9-bdbe-befb34678d15 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.613416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe573721-a03b-4ee9-bdbe-befb34678d15 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.614233 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe573721-a03b-4ee9-bdbe-befb34678d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.641929 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-fe573721-a03b-4ee9-bdbe-befb34678d15 None None] MFA Rules not processed for user `ba49c1c485d64f4f8cdf81d036cf405b`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:35.643175 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fe573721-a03b-4ee9-bdbe-befb34678d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:35.707162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 945/1872] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:35.711703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc59382-ea4e-44d2-834d-ef9566ba7437 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:35.756050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:35.775778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc59382-ea4e-44d2-834d-ef9566ba7437 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:35.780787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7add9a-7708-433d-b067-5b700a7b8e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:35.781426 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7add9a-7708-433d-b067-5b700a7b8e15 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.781703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7add9a-7708-433d-b067-5b700a7b8e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.781972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7add9a-7708-433d-b067-5b700a7b8e15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.782609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7add9a-7708-433d-b067-5b700a7b8e15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:35.784546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7add9a-7708-433d-b067-5b700a7b8e15 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:35.811843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 928/1873] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:35 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:35.815870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe573721-a03b-4ee9-bdbe-befb34678d15 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:35.826811 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe573721-a03b-4ee9-bdbe-befb34678d15 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:35.838719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:35.886843 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ac6c1-5333-4da7-8195-bac1c7ff6138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:35.888175 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ac6c1-5333-4da7-8195-bac1c7ff6138 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.888472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ac6c1-5333-4da7-8195-bac1c7ff6138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.888731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ac6c1-5333-4da7-8195-bac1c7ff6138 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.889371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ac6c1-5333-4da7-8195-bac1c7ff6138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:35.891287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ac6c1-5333-4da7-8195-bac1c7ff6138 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:35.917134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 946/1874] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:35 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:35.918333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7add9a-7708-433d-b067-5b700a7b8e15 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:35.929143 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7add9a-7708-433d-b067-5b700a7b8e15 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:35.934042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e8463-155e-4831-9f72-6faa35e2634a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:35.935609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e8463-155e-4831-9f72-6faa35e2634a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.935938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e8463-155e-4831-9f72-6faa35e2634a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.936214 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e8463-155e-4831-9f72-6faa35e2634a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.936862 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e8463-155e-4831-9f72-6faa35e2634a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:35.938458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e8463-155e-4831-9f72-6faa35e2634a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:35.949970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 929/1875] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:35.953892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ac6c1-5333-4da7-8195-bac1c7ff6138 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:35.965098 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ac6c1-5333-4da7-8195-bac1c7ff6138 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:35.972503 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3a29c-cb9c-461e-8147-d2f379a673e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:35.973170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3a29c-cb9c-461e-8147-d2f379a673e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:35.973452 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3a29c-cb9c-461e-8147-d2f379a673e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:35.973729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3a29c-cb9c-461e-8147-d2f379a673e9 admin admin] PATH_INFO: `/v3/projects/9236b83cf93c4cfa8c494c54b7bcc20d/users/ecec9634ad3a497e921f3e19cba659db/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:35.999289 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3a29c-cb9c-461e-8147-d2f379a673e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9236b83cf93c4cfa8c494c54b7bcc20d, user_id=ecec9634ad3a497e921f3e19cba659db, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:36.007228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3a29c-cb9c-461e-8147-d2f379a673e9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:36.028830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 947/1876] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:35 2022] PUT /identity/v3/projects/9236b83cf93c4cfa8c494c54b7bcc20d/users/ecec9634ad3a497e921f3e19cba659db/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:36.033068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e8463-155e-4831-9f72-6faa35e2634a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:36.045125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e8463-155e-4831-9f72-6faa35e2634a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:36.108887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65393a9b-1fef-4573-84d0-b469f3d0b799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:36.110770 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393a9b-1fef-4573-84d0-b469f3d0b799 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:36.111160 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393a9b-1fef-4573-84d0-b469f3d0b799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:36.111437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393a9b-1fef-4573-84d0-b469f3d0b799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:36.113228 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65393a9b-1fef-4573-84d0-b469f3d0b799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:36.114786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65393a9b-1fef-4573-84d0-b469f3d0b799 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:36.126468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 930/1877] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:36.130410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3a29c-cb9c-461e-8147-d2f379a673e9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:36.141630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3a29c-cb9c-461e-8147-d2f379a673e9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:36.147044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66036e-13f5-423a-a58d-47da218215b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:36.147728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66036e-13f5-423a-a58d-47da218215b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:36.148010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66036e-13f5-423a-a58d-47da218215b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:36.148313 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66036e-13f5-423a-a58d-47da218215b5 admin admin] PATH_INFO: `/v3/projects/9236b83cf93c4cfa8c494c54b7bcc20d/users/ecec9634ad3a497e921f3e19cba659db/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:36.150234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66036e-13f5-423a-a58d-47da218215b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9236b83cf93c4cfa8c494c54b7bcc20d, user_id=ecec9634ad3a497e921f3e19cba659db, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:36.153498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66036e-13f5-423a-a58d-47da218215b5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:36.164241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 948/1878] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:36 2022] PUT /identity/v3/projects/9236b83cf93c4cfa8c494c54b7bcc20d/users/ecec9634ad3a497e921f3e19cba659db/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:36.174655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27039-c4cd-443c-bf80-a6fadd08903b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:36.174940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27039-c4cd-443c-bf80-a6fadd08903b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:36.175252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27039-c4cd-443c-bf80-a6fadd08903b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:36.205840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-10b27039-c4cd-443c-bf80-a6fadd08903b None None] MFA Rules not processed for user `ecec9634ad3a497e921f3e19cba659db`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:36.207068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-10b27039-c4cd-443c-bf80-a6fadd08903b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:36.277986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 931/1879] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:36.290131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66036e-13f5-423a-a58d-47da218215b5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:36.291912 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2a66036e-13f5-423a-a58d-47da218215b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:36.349364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66036e-13f5-423a-a58d-47da218215b5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:36.352987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749aed07-4063-4483-b52e-74f7b27ff5b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:36.353586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749aed07-4063-4483-b52e-74f7b27ff5b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:36.353865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749aed07-4063-4483-b52e-74f7b27ff5b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:36.354131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749aed07-4063-4483-b52e-74f7b27ff5b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:36.363645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749aed07-4063-4483-b52e-74f7b27ff5b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:36.365054 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749aed07-4063-4483-b52e-74f7b27ff5b4 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:36.374562 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 949/1880] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:36.462672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b27039-c4cd-443c-bf80-a6fadd08903b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:36.473079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b27039-c4cd-443c-bf80-a6fadd08903b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:36.476869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95c6e0-cc16-4512-9eae-11630598af71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:36.477522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95c6e0-cc16-4512-9eae-11630598af71 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:36.477855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95c6e0-cc16-4512-9eae-11630598af71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:36.478133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95c6e0-cc16-4512-9eae-11630598af71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:36.513353 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:36.538455 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95c6e0-cc16-4512-9eae-11630598af71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:36.539857 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95c6e0-cc16-4512-9eae-11630598af71 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:36.549400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 932/1881] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:36.823375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749aed07-4063-4483-b52e-74f7b27ff5b4 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:36.885197 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749aed07-4063-4483-b52e-74f7b27ff5b4 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:36.889809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:36.890431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:36.890706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:36.890975 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] PATH_INFO: `/v3/users/ba49c1c485d64f4f8cdf81d036cf405b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:36.892147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba49c1c485d64f4f8cdf81d036cf405b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:36.894052 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:36.908302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:36.941480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba49c1c485d64f4f8cdf81d036cf405b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:36.948599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba49c1c485d64f4f8cdf81d036cf405b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:36.949216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba49c1c485d64f4f8cdf81d036cf405b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:36.961258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba49c1c485d64f4f8cdf81d036cf405b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:36.968889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba49c1c485d64f4f8cdf81d036cf405b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:36.974771 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 950/1882] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:36 2022] DELETE /identity/v3/users/ba49c1c485d64f4f8cdf81d036cf405b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:36.979566 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62337a-7042-47eb-b5f2-66d57ef6e1c2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:36.979858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62337a-7042-47eb-b5f2-66d57ef6e1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:36.980131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62337a-7042-47eb-b5f2-66d57ef6e1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:36.999387 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0b62337a-7042-47eb-b5f2-66d57ef6e1c2 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:37.000602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0b62337a-7042-47eb-b5f2-66d57ef6e1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:37.066736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 933/1883] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:37.076788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:37.078207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:37.138635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a83cc2-787e-4071-b0c4-0bde31d71df3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:37.142726 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b484b-72a8-4ca9-ac9e-0ef1c1b0eff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:37.143403 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b484b-72a8-4ca9-ac9e-0ef1c1b0eff3 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:37.143681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b484b-72a8-4ca9-ac9e-0ef1c1b0eff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:37.143951 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b484b-72a8-4ca9-ac9e-0ef1c1b0eff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:37.157974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b484b-72a8-4ca9-ac9e-0ef1c1b0eff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:37.159879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b484b-72a8-4ca9-ac9e-0ef1c1b0eff3 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:37.171853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 951/1884] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:15:37.550571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62337a-7042-47eb-b5f2-66d57ef6e1c2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:37.552260 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0b62337a-7042-47eb-b5f2-66d57ef6e1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:37.565375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62337a-7042-47eb-b5f2-66d57ef6e1c2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:37.570422 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:37.571119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:37.571405 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:37.571673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] PATH_INFO: `/v3/projects/30e7d8b7dfec48f5b273781e3f4ccdc4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:37.572935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30e7d8b7dfec48f5b273781e3f4ccdc4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:37.575841 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:37.639402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30e7d8b7dfec48f5b273781e3f4ccdc4', 'request_id': 'req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:37.650638 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30e7d8b7dfec48f5b273781e3f4ccdc4', 'request_id': 'req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:37.658979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30e7d8b7dfec48f5b273781e3f4ccdc4', 'request_id': 'req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:37.680531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] The token cache is being invalidate because project 30e7d8b7dfec48f5b273781e3f4ccdc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:37.681177 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:37.682254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:37.685409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 934/1885] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:37 2022] DELETE /identity/v3/projects/30e7d8b7dfec48f5b273781e3f4ccdc4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:37.689339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b484b-72a8-4ca9-ac9e-0ef1c1b0eff3 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:37.699611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b484b-72a8-4ca9-ac9e-0ef1c1b0eff3 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:37.795791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319e02d-df23-4c78-ac47-4f572060ef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:37.796712 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319e02d-df23-4c78-ac47-4f572060ef1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:37.796866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319e02d-df23-4c78-ac47-4f572060ef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:37.797230 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319e02d-df23-4c78-ac47-4f572060ef1c admin admin] PATH_INFO: `/v3/users/ecec9634ad3a497e921f3e19cba659db` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:37.798588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319e02d-df23-4c78-ac47-4f572060ef1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecec9634ad3a497e921f3e19cba659db)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:37.800591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319e02d-df23-4c78-ac47-4f572060ef1c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:37.862883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:37.877296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b319e02d-df23-4c78-ac47-4f572060ef1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecec9634ad3a497e921f3e19cba659db'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:37.883803 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b319e02d-df23-4c78-ac47-4f572060ef1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecec9634ad3a497e921f3e19cba659db'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:37.885084 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b319e02d-df23-4c78-ac47-4f572060ef1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecec9634ad3a497e921f3e19cba659db'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:37.905083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b319e02d-df23-4c78-ac47-4f572060ef1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecec9634ad3a497e921f3e19cba659db'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:37.912549 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b319e02d-df23-4c78-ac47-4f572060ef1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecec9634ad3a497e921f3e19cba659db'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:37.919603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 952/1886] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:37 2022] DELETE /identity/v3/users/ecec9634ad3a497e921f3e19cba659db => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:38.587751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:38.589232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:38.629074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688d0f6-7cf7-48ab-b2bd-1edc8bff5205 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:38.726398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b9111-ffe8-483d-820d-f13cecfe4d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:38.726979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b9111-ffe8-483d-820d-f13cecfe4d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:38.727301 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b9111-ffe8-483d-820d-f13cecfe4d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:38.727571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b9111-ffe8-483d-820d-f13cecfe4d13 admin admin] PATH_INFO: `/v3/projects/9236b83cf93c4cfa8c494c54b7bcc20d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:38.732401 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b9111-ffe8-483d-820d-f13cecfe4d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9236b83cf93c4cfa8c494c54b7bcc20d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:38.734405 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b9111-ffe8-483d-820d-f13cecfe4d13 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:38.736838 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:38.853144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4e3b9111-ffe8-483d-820d-f13cecfe4d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9236b83cf93c4cfa8c494c54b7bcc20d', 'request_id': 'req-4e3b9111-ffe8-483d-820d-f13cecfe4d13'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:38.860893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4e3b9111-ffe8-483d-820d-f13cecfe4d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9236b83cf93c4cfa8c494c54b7bcc20d', 'request_id': 'req-4e3b9111-ffe8-483d-820d-f13cecfe4d13'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:38.868184 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4e3b9111-ffe8-483d-820d-f13cecfe4d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9236b83cf93c4cfa8c494c54b7bcc20d', 'request_id': 'req-4e3b9111-ffe8-483d-820d-f13cecfe4d13'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:38.883828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4e3b9111-ffe8-483d-820d-f13cecfe4d13 admin admin] The token cache is being invalidate because project 9236b83cf93c4cfa8c494c54b7bcc20d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:38.884330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4e3b9111-ffe8-483d-820d-f13cecfe4d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:38.885048 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4e3b9111-ffe8-483d-820d-f13cecfe4d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:38.891489 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 935/1887] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:38 2022] DELETE /identity/v3/projects/9236b83cf93c4cfa8c494c54b7bcc20d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:38.892783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd78fc-1335-4eb3-a83b-52a8418c6146 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:38.893835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd78fc-1335-4eb3-a83b-52a8418c6146 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:38.894364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd78fc-1335-4eb3-a83b-52a8418c6146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:38.922205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e2cd78fc-1335-4eb3-a83b-52a8418c6146 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:38.923673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e2cd78fc-1335-4eb3-a83b-52a8418c6146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:38.985244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 953/1888] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:38.993691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293f17b-6817-45e6-931c-80b0558d6964 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:38.993973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293f17b-6817-45e6-931c-80b0558d6964 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:38.994242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293f17b-6817-45e6-931c-80b0558d6964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:39.015195 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d293f17b-6817-45e6-931c-80b0558d6964 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:39.016442 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d293f17b-6817-45e6-931c-80b0558d6964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:39.024291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 936/1889] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:39.028446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd78fc-1335-4eb3-a83b-52a8418c6146 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:39.037954 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd78fc-1335-4eb3-a83b-52a8418c6146 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:39.043320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53fa1d-25f1-4c29-8e90-df69ce609392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:39.044020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53fa1d-25f1-4c29-8e90-df69ce609392 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:39.044486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53fa1d-25f1-4c29-8e90-df69ce609392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:39.044765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53fa1d-25f1-4c29-8e90-df69ce609392 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:39.045417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53fa1d-25f1-4c29-8e90-df69ce609392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:15:39.045744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53fa1d-25f1-4c29-8e90-df69ce609392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:39.047415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53fa1d-25f1-4c29-8e90-df69ce609392 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:39.058942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 954/1890] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:15:39.064015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293f17b-6817-45e6-931c-80b0558d6964 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:39.073144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293f17b-6817-45e6-931c-80b0558d6964 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:39.079090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df37aa-bcc7-4fc9-9c05-f242ca557b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:39.079687 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df37aa-bcc7-4fc9-9c05-f242ca557b74 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:39.079962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df37aa-bcc7-4fc9-9c05-f242ca557b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:39.080232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df37aa-bcc7-4fc9-9c05-f242ca557b74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:39.080862 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df37aa-bcc7-4fc9-9c05-f242ca557b74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:39.082758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df37aa-bcc7-4fc9-9c05-f242ca557b74 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:39.109579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 937/1891] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:39 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:39.113854 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53fa1d-25f1-4c29-8e90-df69ce609392 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:39.124484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53fa1d-25f1-4c29-8e90-df69ce609392 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:39.189790 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d41d99-bdd3-4600-bd4e-64f9e1b4815b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:39.190470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d41d99-bdd3-4600-bd4e-64f9e1b4815b admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:39.190759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d41d99-bdd3-4600-bd4e-64f9e1b4815b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:39.191028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d41d99-bdd3-4600-bd4e-64f9e1b4815b admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:39.191714 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d41d99-bdd3-4600-bd4e-64f9e1b4815b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:39.193799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d41d99-bdd3-4600-bd4e-64f9e1b4815b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:39.220438 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 955/1892] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:39 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:39.225035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df37aa-bcc7-4fc9-9c05-f242ca557b74 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:39.236735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df37aa-bcc7-4fc9-9c05-f242ca557b74 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:39.241834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5c1a8-00e4-496a-bf39-e02cf58e7be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:39.242536 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5c1a8-00e4-496a-bf39-e02cf58e7be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:39.242877 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5c1a8-00e4-496a-bf39-e02cf58e7be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:39.243250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5c1a8-00e4-496a-bf39-e02cf58e7be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:39.243960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5c1a8-00e4-496a-bf39-e02cf58e7be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:39.245664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5c1a8-00e4-496a-bf39-e02cf58e7be3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:39.260233 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 938/1893] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:15:39.261528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d41d99-bdd3-4600-bd4e-64f9e1b4815b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:39.274150 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d41d99-bdd3-4600-bd4e-64f9e1b4815b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:39.282542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d75323-991a-4671-9366-57fa6c6ff54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:39.283481 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d75323-991a-4671-9366-57fa6c6ff54a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:39.283611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d75323-991a-4671-9366-57fa6c6ff54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:39.285356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d75323-991a-4671-9366-57fa6c6ff54a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:39.286498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d75323-991a-4671-9366-57fa6c6ff54a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:39.288283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d75323-991a-4671-9366-57fa6c6ff54a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:39.302259 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 956/1894] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:39.306291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5c1a8-00e4-496a-bf39-e02cf58e7be3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:39.321059 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5c1a8-00e4-496a-bf39-e02cf58e7be3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:39.329015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e38fa-03be-4d6f-af98-7b0fe5e49d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:39.329703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e38fa-03be-4d6f-af98-7b0fe5e49d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:39.329981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e38fa-03be-4d6f-af98-7b0fe5e49d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:39.330394 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e38fa-03be-4d6f-af98-7b0fe5e49d39 admin admin] PATH_INFO: `/v3/projects/ca38013a20eb4c8e830e391cc06f485f/users/d73fdae39484489facaf6831430cd4d7/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:39.348714 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e38fa-03be-4d6f-af98-7b0fe5e49d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca38013a20eb4c8e830e391cc06f485f, user_id=d73fdae39484489facaf6831430cd4d7, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:39.352004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e38fa-03be-4d6f-af98-7b0fe5e49d39 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:39.378905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 939/1895] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:39 2022] PUT /identity/v3/projects/ca38013a20eb4c8e830e391cc06f485f/users/d73fdae39484489facaf6831430cd4d7/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:39.382520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53db160-95ba-40b1-9804-33f42ae5ffe0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:39.382973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53db160-95ba-40b1-9804-33f42ae5ffe0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:39.383275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53db160-95ba-40b1-9804-33f42ae5ffe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:39.404505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a53db160-95ba-40b1-9804-33f42ae5ffe0 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:39.406240 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a53db160-95ba-40b1-9804-33f42ae5ffe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:39.424501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:39.494363 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 957/1896] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:39.507549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e38fa-03be-4d6f-af98-7b0fe5e49d39 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:39.509478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-181e38fa-03be-4d6f-af98-7b0fe5e49d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:39.574463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e38fa-03be-4d6f-af98-7b0fe5e49d39 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:39.580498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0439933-d01d-44cc-806d-5a11534b158b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:39.581219 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0439933-d01d-44cc-806d-5a11534b158b service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:39.581504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0439933-d01d-44cc-806d-5a11534b158b service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:39.581781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0439933-d01d-44cc-806d-5a11534b158b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:39.597038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0439933-d01d-44cc-806d-5a11534b158b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:39.599231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0439933-d01d-44cc-806d-5a11534b158b service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:39.611881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 940/1897] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:15:40.474116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53db160-95ba-40b1-9804-33f42ae5ffe0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:40.475684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a53db160-95ba-40b1-9804-33f42ae5ffe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:40.500605 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53db160-95ba-40b1-9804-33f42ae5ffe0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:40.506758 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f5ae8-0d17-4178-a9a6-9cc26693c980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:40.507464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f5ae8-0d17-4178-a9a6-9cc26693c980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:40.507743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f5ae8-0d17-4178-a9a6-9cc26693c980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:40.508015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f5ae8-0d17-4178-a9a6-9cc26693c980 admin admin] PATH_INFO: `/v3/users/fd1a7ec2da294f3a92b061a87b306a19` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:40.509362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f5ae8-0d17-4178-a9a6-9cc26693c980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd1a7ec2da294f3a92b061a87b306a19)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:40.521094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f5ae8-0d17-4178-a9a6-9cc26693c980 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:40.645234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-841f5ae8-0d17-4178-a9a6-9cc26693c980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd1a7ec2da294f3a92b061a87b306a19'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:40.664626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-841f5ae8-0d17-4178-a9a6-9cc26693c980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd1a7ec2da294f3a92b061a87b306a19'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:40.665345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-841f5ae8-0d17-4178-a9a6-9cc26693c980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd1a7ec2da294f3a92b061a87b306a19'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:40.686192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-841f5ae8-0d17-4178-a9a6-9cc26693c980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd1a7ec2da294f3a92b061a87b306a19'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:40.695826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-841f5ae8-0d17-4178-a9a6-9cc26693c980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd1a7ec2da294f3a92b061a87b306a19'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:40.707529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 958/1898] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:40 2022] DELETE /identity/v3/users/fd1a7ec2da294f3a92b061a87b306a19 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:40.712110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf2eaf-2682-4e7a-bba9-33d5b62b1937 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:40.712615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf2eaf-2682-4e7a-bba9-33d5b62b1937 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:40.712695 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf2eaf-2682-4e7a-bba9-33d5b62b1937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:40.735371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d6cf2eaf-2682-4e7a-bba9-33d5b62b1937 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:40.736652 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d6cf2eaf-2682-4e7a-bba9-33d5b62b1937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:40.769025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:40.804504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1835c-9a84-49b7-b151-951760db4156 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:40.804788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1835c-9a84-49b7-b151-951760db4156 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:40.805065 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1835c-9a84-49b7-b151-951760db4156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:40.812380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 941/1899] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:40.822467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf2eaf-2682-4e7a-bba9-33d5b62b1937 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:40.824537 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d6cf2eaf-2682-4e7a-bba9-33d5b62b1937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:40.838986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-70e1835c-9a84-49b7-b151-951760db4156 None None] MFA Rules not processed for user `d73fdae39484489facaf6831430cd4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:40.841120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-70e1835c-9a84-49b7-b151-951760db4156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:40.922439 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf2eaf-2682-4e7a-bba9-33d5b62b1937 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:40.922639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 959/1900] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:40.930374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92921391-5815-466f-a7e1-4d8131e85361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:40.931932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92921391-5815-466f-a7e1-4d8131e85361 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:40.932842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1835c-9a84-49b7-b151-951760db4156 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:40.934340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-70e1835c-9a84-49b7-b151-951760db4156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:40.934765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92921391-5815-466f-a7e1-4d8131e85361 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:40.935427 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92921391-5815-466f-a7e1-4d8131e85361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:40.955955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92921391-5815-466f-a7e1-4d8131e85361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:40.957876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92921391-5815-466f-a7e1-4d8131e85361 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:40.968859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:40.972413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 942/1901] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:15:41.008866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1835c-9a84-49b7-b151-951760db4156 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:41.015093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d2854-5900-4c56-aaab-b821b67afba7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:41.015262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d2854-5900-4c56-aaab-b821b67afba7 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:41.015498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d2854-5900-4c56-aaab-b821b67afba7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:41.015767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d2854-5900-4c56-aaab-b821b67afba7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:41.026469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d2854-5900-4c56-aaab-b821b67afba7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:41.028196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d2854-5900-4c56-aaab-b821b67afba7 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:41.039304 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 960/1902] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:41.292979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92921391-5815-466f-a7e1-4d8131e85361 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:41.295564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-92921391-5815-466f-a7e1-4d8131e85361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:41.328969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92921391-5815-466f-a7e1-4d8131e85361 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:41.334408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:41.335040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:41.335520 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:41.336134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] PATH_INFO: `/v3/projects/17775293d617489c8d24f758ad5142ce` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:41.337401 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=17775293d617489c8d24f758ad5142ce)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:41.339655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:41.400664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17775293d617489c8d24f758ad5142ce', 'request_id': 'req-46aa4b6f-a066-424c-9fd4-08e30275cd2b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:41.407979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17775293d617489c8d24f758ad5142ce', 'request_id': 'req-46aa4b6f-a066-424c-9fd4-08e30275cd2b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:41.415412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17775293d617489c8d24f758ad5142ce', 'request_id': 'req-46aa4b6f-a066-424c-9fd4-08e30275cd2b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:41.430966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] The token cache is being invalidate because project 17775293d617489c8d24f758ad5142ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:41.431400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:41.432158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:41.433743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 943/1903] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:41 2022] DELETE /identity/v3/projects/17775293d617489c8d24f758ad5142ce => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:41.438372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d2854-5900-4c56-aaab-b821b67afba7 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:41.449677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d2854-5900-4c56-aaab-b821b67afba7 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:41.524476 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399f58b-fa40-4f97-a9bc-9fd66783c9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:41.525128 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399f58b-fa40-4f97-a9bc-9fd66783c9d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:41.525405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399f58b-fa40-4f97-a9bc-9fd66783c9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:41.525672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399f58b-fa40-4f97-a9bc-9fd66783c9d7 admin admin] PATH_INFO: `/v3/users/41db780aa8954fbcbd5f8cdf820707b8` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:41.526994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399f58b-fa40-4f97-a9bc-9fd66783c9d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41db780aa8954fbcbd5f8cdf820707b8)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:41.530207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399f58b-fa40-4f97-a9bc-9fd66783c9d7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:41.581509 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8399f58b-fa40-4f97-a9bc-9fd66783c9d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41db780aa8954fbcbd5f8cdf820707b8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:41.588878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8399f58b-fa40-4f97-a9bc-9fd66783c9d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41db780aa8954fbcbd5f8cdf820707b8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:41.589532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8399f58b-fa40-4f97-a9bc-9fd66783c9d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41db780aa8954fbcbd5f8cdf820707b8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:41.620409 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8399f58b-fa40-4f97-a9bc-9fd66783c9d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41db780aa8954fbcbd5f8cdf820707b8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:41.629835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8399f58b-fa40-4f97-a9bc-9fd66783c9d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41db780aa8954fbcbd5f8cdf820707b8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:41.638788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 961/1904] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:41 2022] DELETE /identity/v3/users/41db780aa8954fbcbd5f8cdf820707b8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:41.994212 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:41.996577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:42.021028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa4b6f-a066-424c-9fd4-08e30275cd2b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:42.072854 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:42.081132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:42.081931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:42.082573 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:42.082933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852 admin admin] PATH_INFO: `/v3/projects/745bdeeed0ab43bbbdc84f0bf212826c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:42.084216 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=745bdeeed0ab43bbbdc84f0bf212826c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:42.086217 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:42.134213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '745bdeeed0ab43bbbdc84f0bf212826c', 'request_id': 'req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:42.140286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '745bdeeed0ab43bbbdc84f0bf212826c', 'request_id': 'req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:42.146525 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '745bdeeed0ab43bbbdc84f0bf212826c', 'request_id': 'req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:42.162871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852 admin admin] The token cache is being invalidate because project 745bdeeed0ab43bbbdc84f0bf212826c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:42.163188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:42.163812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-986b6e29-8ee2-425d-9c93-9b5a8c9c9852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:42.165638 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 944/1905] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:41 2022] DELETE /identity/v3/projects/745bdeeed0ab43bbbdc84f0bf212826c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:42.170580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162cacc-8e2e-45a9-bfaf-817d4e213329 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:42.170865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162cacc-8e2e-45a9-bfaf-817d4e213329 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:42.171169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162cacc-8e2e-45a9-bfaf-817d4e213329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:42.191575 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4162cacc-8e2e-45a9-bfaf-817d4e213329 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:42.192801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4162cacc-8e2e-45a9-bfaf-817d4e213329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:42.238160 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:42.258732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 962/1906] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:42.267159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2786c84-e51e-4a43-9431-e346f80008b5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:42.267445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2786c84-e51e-4a43-9431-e346f80008b5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:42.267718 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2786c84-e51e-4a43-9431-e346f80008b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:42.288148 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f2786c84-e51e-4a43-9431-e346f80008b5 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:42.289347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f2786c84-e51e-4a43-9431-e346f80008b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:42.298276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 945/1907] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:42.302629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162cacc-8e2e-45a9-bfaf-817d4e213329 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:42.312629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162cacc-8e2e-45a9-bfaf-817d4e213329 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:42.317273 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32c687-80f8-41f2-84c5-e86da9c8c478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:42.317877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32c687-80f8-41f2-84c5-e86da9c8c478 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:42.318157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32c687-80f8-41f2-84c5-e86da9c8c478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:42.318430 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32c687-80f8-41f2-84c5-e86da9c8c478 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:42.319117 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32c687-80f8-41f2-84c5-e86da9c8c478 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:15:42.319427 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32c687-80f8-41f2-84c5-e86da9c8c478 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:42.321022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32c687-80f8-41f2-84c5-e86da9c8c478 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:42.332995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 963/1908] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:15:42.337153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2786c84-e51e-4a43-9431-e346f80008b5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:42.346982 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2786c84-e51e-4a43-9431-e346f80008b5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:42.351685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cc12e-24e5-4f1f-8ad3-7a3016affab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:42.352281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cc12e-24e5-4f1f-8ad3-7a3016affab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:42.352557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cc12e-24e5-4f1f-8ad3-7a3016affab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:42.352825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cc12e-24e5-4f1f-8ad3-7a3016affab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:42.353463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cc12e-24e5-4f1f-8ad3-7a3016affab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:42.355407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cc12e-24e5-4f1f-8ad3-7a3016affab1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:42.384436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 946/1909] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:42 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:42.389323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32c687-80f8-41f2-84c5-e86da9c8c478 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:42.400151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32c687-80f8-41f2-84c5-e86da9c8c478 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:42.461954 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c86fc-829c-4c32-8cc0-c3cd872d0208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:42.462561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c86fc-829c-4c32-8cc0-c3cd872d0208 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:42.462840 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c86fc-829c-4c32-8cc0-c3cd872d0208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:42.463142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c86fc-829c-4c32-8cc0-c3cd872d0208 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:42.463761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c86fc-829c-4c32-8cc0-c3cd872d0208 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:42.466390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c86fc-829c-4c32-8cc0-c3cd872d0208 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:42.488521 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 964/1910] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:42 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:42.492673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cc12e-24e5-4f1f-8ad3-7a3016affab1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:42.503812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cc12e-24e5-4f1f-8ad3-7a3016affab1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:42.508819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44121f0d-ca71-4bc9-a0b6-51400f87751a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:42.509368 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44121f0d-ca71-4bc9-a0b6-51400f87751a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:42.509641 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44121f0d-ca71-4bc9-a0b6-51400f87751a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:42.509909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44121f0d-ca71-4bc9-a0b6-51400f87751a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:42.510543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44121f0d-ca71-4bc9-a0b6-51400f87751a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:42.512152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44121f0d-ca71-4bc9-a0b6-51400f87751a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:42.523190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 947/1911] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:42.527300 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c86fc-829c-4c32-8cc0-c3cd872d0208 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:42.538177 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c86fc-829c-4c32-8cc0-c3cd872d0208 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:42.542829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b634a67-d5b2-4aeb-9224-3cea9c903112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:42.543508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b634a67-d5b2-4aeb-9224-3cea9c903112 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:42.543755 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b634a67-d5b2-4aeb-9224-3cea9c903112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:42.544029 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b634a67-d5b2-4aeb-9224-3cea9c903112 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:42.544684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b634a67-d5b2-4aeb-9224-3cea9c903112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:42.546353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b634a67-d5b2-4aeb-9224-3cea9c903112 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:42.560173 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 965/1912] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:42.567191 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44121f0d-ca71-4bc9-a0b6-51400f87751a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:42.578355 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44121f0d-ca71-4bc9-a0b6-51400f87751a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:42.583668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60391326-be0f-43a7-8a1b-c2b2758d1424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:42.584448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60391326-be0f-43a7-8a1b-c2b2758d1424 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:42.584820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60391326-be0f-43a7-8a1b-c2b2758d1424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:42.585162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60391326-be0f-43a7-8a1b-c2b2758d1424 admin admin] PATH_INFO: `/v3/projects/c37c01eadcbb4ad79af94c1950000f44/users/89aa7982d1ee4c058e361e77c065d705/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:42.605720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60391326-be0f-43a7-8a1b-c2b2758d1424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37c01eadcbb4ad79af94c1950000f44, user_id=89aa7982d1ee4c058e361e77c065d705, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:42.608715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60391326-be0f-43a7-8a1b-c2b2758d1424 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:42.624714 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 948/1913] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:42 2022] PUT /identity/v3/projects/c37c01eadcbb4ad79af94c1950000f44/users/89aa7982d1ee4c058e361e77c065d705/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:42.636021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f9daa-a3e1-40d5-9825-70e72349a482 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:42.636309 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f9daa-a3e1-40d5-9825-70e72349a482 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:42.636585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f9daa-a3e1-40d5-9825-70e72349a482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:42.666708 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-825f9daa-a3e1-40d5-9825-70e72349a482 None None] MFA Rules not processed for user `89aa7982d1ee4c058e361e77c065d705`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:42.668191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-825f9daa-a3e1-40d5-9825-70e72349a482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:42.744671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 966/1914] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:42.749281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60391326-be0f-43a7-8a1b-c2b2758d1424 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:42.810873 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60391326-be0f-43a7-8a1b-c2b2758d1424 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:42.815474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e12d75-146c-4ac7-ba92-900d6bd6c49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:42.816085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e12d75-146c-4ac7-ba92-900d6bd6c49b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:42.816365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e12d75-146c-4ac7-ba92-900d6bd6c49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:42.816638 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e12d75-146c-4ac7-ba92-900d6bd6c49b admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:42.817271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e12d75-146c-4ac7-ba92-900d6bd6c49b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:42.819232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e12d75-146c-4ac7-ba92-900d6bd6c49b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:42.846969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 949/1915] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:42 2022] POST /identity/v3/projects => generated 391 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:42.850833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f9daa-a3e1-40d5-9825-70e72349a482 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:42.862703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f9daa-a3e1-40d5-9825-70e72349a482 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:42.901126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:42.922313 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c1b64-b292-436d-8c94-b31ee30a443f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:42.922916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c1b64-b292-436d-8c94-b31ee30a443f admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:42.923229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c1b64-b292-436d-8c94-b31ee30a443f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:42.923495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c1b64-b292-436d-8c94-b31ee30a443f admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:42.924161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c1b64-b292-436d-8c94-b31ee30a443f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:42.926057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c1b64-b292-436d-8c94-b31ee30a443f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:42.946584 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 967/1916] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:42 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:42.951195 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e12d75-146c-4ac7-ba92-900d6bd6c49b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:42.961584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e12d75-146c-4ac7-ba92-900d6bd6c49b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:42.967183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ecef0-2a92-4109-a59a-ed2d75f10bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:42.967796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ecef0-2a92-4109-a59a-ed2d75f10bdb admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:42.968086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ecef0-2a92-4109-a59a-ed2d75f10bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:42.968366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ecef0-2a92-4109-a59a-ed2d75f10bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:42.969016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ecef0-2a92-4109-a59a-ed2d75f10bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:42.970616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ecef0-2a92-4109-a59a-ed2d75f10bdb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:42.971986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:42.984072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 950/1917] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:42.988774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c1b64-b292-436d-8c94-b31ee30a443f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:42.998973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c1b64-b292-436d-8c94-b31ee30a443f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:43.004128 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb44b0-7f43-4ac7-8a7d-09a46f43065e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:43.004781 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb44b0-7f43-4ac7-8a7d-09a46f43065e admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:43.005074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb44b0-7f43-4ac7-8a7d-09a46f43065e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:43.005382 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb44b0-7f43-4ac7-8a7d-09a46f43065e admin admin] PATH_INFO: `/v3/projects/7faeeaa7a4ee47ab96e6c885ed4384d3/users/ad34700e2a564060b175f8dbd16521e7/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:43.022567 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb44b0-7f43-4ac7-8a7d-09a46f43065e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7faeeaa7a4ee47ab96e6c885ed4384d3, user_id=ad34700e2a564060b175f8dbd16521e7, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:43.025281 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb44b0-7f43-4ac7-8a7d-09a46f43065e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:43.045073 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 968/1918] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:42 2022] PUT /identity/v3/projects/7faeeaa7a4ee47ab96e6c885ed4384d3/users/ad34700e2a564060b175f8dbd16521e7/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:43.048223 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ecef0-2a92-4109-a59a-ed2d75f10bdb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:43.059289 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ecef0-2a92-4109-a59a-ed2d75f10bdb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:43.124936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670b9c9-965b-46e5-aa87-2a64a16037f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:43.125649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670b9c9-965b-46e5-aa87-2a64a16037f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:43.125935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670b9c9-965b-46e5-aa87-2a64a16037f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:43.126237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670b9c9-965b-46e5-aa87-2a64a16037f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:43.127094 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670b9c9-965b-46e5-aa87-2a64a16037f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:43.128617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670b9c9-965b-46e5-aa87-2a64a16037f6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:43.141527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 951/1919] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:43.145882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb44b0-7f43-4ac7-8a7d-09a46f43065e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:43.157312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb44b0-7f43-4ac7-8a7d-09a46f43065e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:43.162623 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074be752-565f-40e9-bf49-68c5ae557a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:43.163344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074be752-565f-40e9-bf49-68c5ae557a69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:43.163629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074be752-565f-40e9-bf49-68c5ae557a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:43.163898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074be752-565f-40e9-bf49-68c5ae557a69 admin admin] PATH_INFO: `/v3/projects/7faeeaa7a4ee47ab96e6c885ed4384d3/users/ad34700e2a564060b175f8dbd16521e7/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:43.166006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074be752-565f-40e9-bf49-68c5ae557a69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7faeeaa7a4ee47ab96e6c885ed4384d3, user_id=ad34700e2a564060b175f8dbd16521e7, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:43.169217 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074be752-565f-40e9-bf49-68c5ae557a69 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:43.180056 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 969/1920] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:43 2022] PUT /identity/v3/projects/7faeeaa7a4ee47ab96e6c885ed4384d3/users/ad34700e2a564060b175f8dbd16521e7/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:43.190823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9037f-db72-496f-99fc-2fe6fdf992f9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:43.191146 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9037f-db72-496f-99fc-2fe6fdf992f9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:43.191422 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9037f-db72-496f-99fc-2fe6fdf992f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:43.220811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-56a9037f-db72-496f-99fc-2fe6fdf992f9 None None] MFA Rules not processed for user `ad34700e2a564060b175f8dbd16521e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:43.222038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-56a9037f-db72-496f-99fc-2fe6fdf992f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:43.291661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 952/1921] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:43 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:43.302440 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074be752-565f-40e9-bf49-68c5ae557a69 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:43.304249 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-074be752-565f-40e9-bf49-68c5ae557a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:43.363163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074be752-565f-40e9-bf49-68c5ae557a69 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:43.366936 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a293c-fcbd-487c-81a2-390800bd2e84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:43.367582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a293c-fcbd-487c-81a2-390800bd2e84 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:43.367860 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a293c-fcbd-487c-81a2-390800bd2e84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:43.368129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a293c-fcbd-487c-81a2-390800bd2e84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:43.377746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a293c-fcbd-487c-81a2-390800bd2e84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:43.379288 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a293c-fcbd-487c-81a2-390800bd2e84 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:43.388906 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 970/1922] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:43 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:43.436254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9037f-db72-496f-99fc-2fe6fdf992f9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:43.437647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-56a9037f-db72-496f-99fc-2fe6fdf992f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:43.449421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9037f-db72-496f-99fc-2fe6fdf992f9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:43.453686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ae0d7-aca8-4b84-bc3c-2a0040f358b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:43.454302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ae0d7-aca8-4b84-bc3c-2a0040f358b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:43.454580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ae0d7-aca8-4b84-bc3c-2a0040f358b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:43.454855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ae0d7-aca8-4b84-bc3c-2a0040f358b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:43.517508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ae0d7-aca8-4b84-bc3c-2a0040f358b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:43.519065 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ae0d7-aca8-4b84-bc3c-2a0040f358b0 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:43.529613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 953/1923] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:43 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:44.786264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844e0db-f09e-47a4-91aa-2c83d1ac15f0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:44.786766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844e0db-f09e-47a4-91aa-2c83d1ac15f0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:44.786972 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844e0db-f09e-47a4-91aa-2c83d1ac15f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:44.814702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e844e0db-f09e-47a4-91aa-2c83d1ac15f0 None None] MFA Rules not processed for user `d73fdae39484489facaf6831430cd4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:44.817014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e844e0db-f09e-47a4-91aa-2c83d1ac15f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:44.927122 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 971/1924] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:44.979229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ae0d7-aca8-4b84-bc3c-2a0040f358b0 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:44.989921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ae0d7-aca8-4b84-bc3c-2a0040f358b0 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:44.997306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f01d0-e86f-4601-a643-3549ba4f3f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:45.001190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f01d0-e86f-4601-a643-3549ba4f3f3e service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:45.001539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f01d0-e86f-4601-a643-3549ba4f3f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:45.001883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f01d0-e86f-4601-a643-3549ba4f3f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:45.011144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c51f01d0-e86f-4601-a643-3549ba4f3f3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:45.044320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:45.119273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f01d0-e86f-4601-a643-3549ba4f3f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:45.121018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f01d0-e86f-4601-a643-3549ba4f3f3e service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:45.146963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 954/1925] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:47.315120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844e0db-f09e-47a4-91aa-2c83d1ac15f0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:47.363240 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:47.386022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844e0db-f09e-47a4-91aa-2c83d1ac15f0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:47.391641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:47.392389 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:47.392743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:47.393022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] PATH_INFO: `/v3/users/89aa7982d1ee4c058e361e77c065d705` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:47.394170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89aa7982d1ee4c058e361e77c065d705)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:47.396177 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:47.457334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89aa7982d1ee4c058e361e77c065d705'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:47.467346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89aa7982d1ee4c058e361e77c065d705'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:47.468007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89aa7982d1ee4c058e361e77c065d705'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:47.496034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89aa7982d1ee4c058e361e77c065d705'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:47.508778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89aa7982d1ee4c058e361e77c065d705'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:47.519501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 972/1926] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:47 2022] DELETE /identity/v3/users/89aa7982d1ee4c058e361e77c065d705 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:47.524281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63610fcf-2add-4d00-b293-1af7f0699a14 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:47.525278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63610fcf-2add-4d00-b293-1af7f0699a14 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:47.525923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63610fcf-2add-4d00-b293-1af7f0699a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:47.583291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-63610fcf-2add-4d00-b293-1af7f0699a14 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:47.585698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-63610fcf-2add-4d00-b293-1af7f0699a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:47.703546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 955/1927] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:47.715013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:47.717332 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:47.823861 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780099d-9a1f-4b68-95c0-f818f9f80471 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:47.827682 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f43f9d-38e6-4a21-ae95-fd834c7a0386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:47.828413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f43f9d-38e6-4a21-ae95-fd834c7a0386 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:47.828741 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f43f9d-38e6-4a21-ae95-fd834c7a0386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:47.829083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f43f9d-38e6-4a21-ae95-fd834c7a0386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:47.844966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f43f9d-38e6-4a21-ae95-fd834c7a0386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:47.849043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f43f9d-38e6-4a21-ae95-fd834c7a0386 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:47.869533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 973/1928] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:48.576525 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63610fcf-2add-4d00-b293-1af7f0699a14 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:48.578022 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-63610fcf-2add-4d00-b293-1af7f0699a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:48.638806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63610fcf-2add-4d00-b293-1af7f0699a14 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:48.670251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:48.671089 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:48.671478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:48.673892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] PATH_INFO: `/v3/projects/c37c01eadcbb4ad79af94c1950000f44` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:48.684861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c37c01eadcbb4ad79af94c1950000f44)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:48.691164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:48.717930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f43f9d-38e6-4a21-ae95-fd834c7a0386 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:48.719287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b9f43f9d-38e6-4a21-ae95-fd834c7a0386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:48.746064 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f43f9d-38e6-4a21-ae95-fd834c7a0386 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:48.764838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f39d98-ed8e-42bb-9e79-646cc0156924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:48.765621 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f39d98-ed8e-42bb-9e79-646cc0156924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:48.766742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f39d98-ed8e-42bb-9e79-646cc0156924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:48.767166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f39d98-ed8e-42bb-9e79-646cc0156924 admin admin] PATH_INFO: `/v3/users/e56ff1ab4e5b44b8b295a073979c1edd` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:48.769705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f39d98-ed8e-42bb-9e79-646cc0156924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e56ff1ab4e5b44b8b295a073979c1edd)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:48.771961 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f39d98-ed8e-42bb-9e79-646cc0156924 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:48.860495 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c37c01eadcbb4ad79af94c1950000f44', 'request_id': 'req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:48.873978 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5f39d98-ed8e-42bb-9e79-646cc0156924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e56ff1ab4e5b44b8b295a073979c1edd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:48.895588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5f39d98-ed8e-42bb-9e79-646cc0156924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e56ff1ab4e5b44b8b295a073979c1edd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:48.896494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5f39d98-ed8e-42bb-9e79-646cc0156924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e56ff1ab4e5b44b8b295a073979c1edd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:48.898590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c37c01eadcbb4ad79af94c1950000f44', 'request_id': 'req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:48.899885 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:48.918006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5f39d98-ed8e-42bb-9e79-646cc0156924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e56ff1ab4e5b44b8b295a073979c1edd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:48.922218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c37c01eadcbb4ad79af94c1950000f44', 'request_id': 'req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:48.929293 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5f39d98-ed8e-42bb-9e79-646cc0156924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e56ff1ab4e5b44b8b295a073979c1edd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:48.939187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 974/1929] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:48 2022] DELETE /identity/v3/users/e56ff1ab4e5b44b8b295a073979c1edd => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:48.944170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa647fc-3a36-4928-b2b6-bc0f9f71cab6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:48.944599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa647fc-3a36-4928-b2b6-bc0f9f71cab6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:48.944972 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa647fc-3a36-4928-b2b6-bc0f9f71cab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:48.952545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] The token cache is being invalidate because project c37c01eadcbb4ad79af94c1950000f44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:48.952854 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:48.954604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:48.963373 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 956/1930] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:48 2022] DELETE /identity/v3/projects/c37c01eadcbb4ad79af94c1950000f44 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:48.964514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:48.966674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:48.976270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dfa647fc-3a36-4928-b2b6-bc0f9f71cab6 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:48.977830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dfa647fc-3a36-4928-b2b6-bc0f9f71cab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:49.014087 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:49.067372 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113b523-6d48-4e4b-81ae-d91d3d54b8c5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:49.085506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 975/1931] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:49.106790 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa647fc-3a36-4928-b2b6-bc0f9f71cab6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:49.108913 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dfa647fc-3a36-4928-b2b6-bc0f9f71cab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:49.186866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc4527-7df7-436a-a5ff-fdb799ed8b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:49.187793 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc4527-7df7-436a-a5ff-fdb799ed8b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:49.190350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc4527-7df7-436a-a5ff-fdb799ed8b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:49.191777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc4527-7df7-436a-a5ff-fdb799ed8b79 admin admin] PATH_INFO: `/v3/users/ad34700e2a564060b175f8dbd16521e7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:49.195101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc4527-7df7-436a-a5ff-fdb799ed8b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad34700e2a564060b175f8dbd16521e7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:49.200710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc4527-7df7-436a-a5ff-fdb799ed8b79 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:49.235733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa647fc-3a36-4928-b2b6-bc0f9f71cab6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:49.251479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57ef9f-ea30-48f7-bfaa-b92804995afc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:49.252694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57ef9f-ea30-48f7-bfaa-b92804995afc service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:49.252981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57ef9f-ea30-48f7-bfaa-b92804995afc service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:49.253256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57ef9f-ea30-48f7-bfaa-b92804995afc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:49.269214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a57ef9f-ea30-48f7-bfaa-b92804995afc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:49.270556 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a57ef9f-ea30-48f7-bfaa-b92804995afc service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:49.282840 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 976/1932] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:49.322364 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0fdc4527-7df7-436a-a5ff-fdb799ed8b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad34700e2a564060b175f8dbd16521e7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:49.338513 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0fdc4527-7df7-436a-a5ff-fdb799ed8b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad34700e2a564060b175f8dbd16521e7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:49.367533 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0fdc4527-7df7-436a-a5ff-fdb799ed8b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad34700e2a564060b175f8dbd16521e7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:49.379084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0fdc4527-7df7-436a-a5ff-fdb799ed8b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad34700e2a564060b175f8dbd16521e7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:49.384409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0fdc4527-7df7-436a-a5ff-fdb799ed8b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad34700e2a564060b175f8dbd16521e7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:49.407148 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 957/1933] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:48 2022] DELETE /identity/v3/users/ad34700e2a564060b175f8dbd16521e7 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:49.758684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57ef9f-ea30-48f7-bfaa-b92804995afc service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:49.777851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57ef9f-ea30-48f7-bfaa-b92804995afc service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:49.872652 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93112e1f-e943-4cae-9f91-d795d34aea21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:49.873631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93112e1f-e943-4cae-9f91-d795d34aea21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:49.874041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93112e1f-e943-4cae-9f91-d795d34aea21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:49.874414 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93112e1f-e943-4cae-9f91-d795d34aea21 admin admin] PATH_INFO: `/v3/projects/7faeeaa7a4ee47ab96e6c885ed4384d3` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:49.876812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93112e1f-e943-4cae-9f91-d795d34aea21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7faeeaa7a4ee47ab96e6c885ed4384d3)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:49.881346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93112e1f-e943-4cae-9f91-d795d34aea21 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:49.915721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:49.953563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-93112e1f-e943-4cae-9f91-d795d34aea21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7faeeaa7a4ee47ab96e6c885ed4384d3', 'request_id': 'req-93112e1f-e943-4cae-9f91-d795d34aea21'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:49.963772 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-93112e1f-e943-4cae-9f91-d795d34aea21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7faeeaa7a4ee47ab96e6c885ed4384d3', 'request_id': 'req-93112e1f-e943-4cae-9f91-d795d34aea21'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:49.974680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-93112e1f-e943-4cae-9f91-d795d34aea21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7faeeaa7a4ee47ab96e6c885ed4384d3', 'request_id': 'req-93112e1f-e943-4cae-9f91-d795d34aea21'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:50.003605 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-93112e1f-e943-4cae-9f91-d795d34aea21 admin admin] The token cache is being invalidate because project 7faeeaa7a4ee47ab96e6c885ed4384d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:50.004049 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-93112e1f-e943-4cae-9f91-d795d34aea21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:50.005055 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-93112e1f-e943-4cae-9f91-d795d34aea21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:50.007922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 977/1934] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:49 2022] DELETE /identity/v3/projects/7faeeaa7a4ee47ab96e6c885ed4384d3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:50.012847 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77805ed9-4654-46b3-804f-6ef3ece38dd5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:50.013407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77805ed9-4654-46b3-804f-6ef3ece38dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:50.013407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77805ed9-4654-46b3-804f-6ef3ece38dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:50.050051 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-77805ed9-4654-46b3-804f-6ef3ece38dd5 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:50.051796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-77805ed9-4654-46b3-804f-6ef3ece38dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:50.130225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:50.148276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 958/1935] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:50.149215 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a1e3-a996-45cd-a636-de83e19cec9d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:50.149663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a1e3-a996-45cd-a636-de83e19cec9d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:50.149771 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a1e3-a996-45cd-a636-de83e19cec9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:50.170973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d5d5a1e3-a996-45cd-a636-de83e19cec9d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:50.172304 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d5d5a1e3-a996-45cd-a636-de83e19cec9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:50.184626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 978/1936] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:50.192638 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77805ed9-4654-46b3-804f-6ef3ece38dd5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:50.208192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77805ed9-4654-46b3-804f-6ef3ece38dd5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:50.216915 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2069a-0afa-4f82-a476-8e8a07371b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:50.217753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2069a-0afa-4f82-a476-8e8a07371b88 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:50.217955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2069a-0afa-4f82-a476-8e8a07371b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:50.218232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2069a-0afa-4f82-a476-8e8a07371b88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:50.219483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2069a-0afa-4f82-a476-8e8a07371b88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:15:50.219835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2069a-0afa-4f82-a476-8e8a07371b88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:50.221561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2069a-0afa-4f82-a476-8e8a07371b88 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:50.241775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 959/1937] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:15:50.242362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5a1e3-a996-45cd-a636-de83e19cec9d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:50.251709 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5a1e3-a996-45cd-a636-de83e19cec9d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:50.257182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ce0c1-cb8a-4390-a4a6-8cc49a664356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:50.257824 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ce0c1-cb8a-4390-a4a6-8cc49a664356 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:50.258086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ce0c1-cb8a-4390-a4a6-8cc49a664356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:50.258358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ce0c1-cb8a-4390-a4a6-8cc49a664356 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:50.259006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ce0c1-cb8a-4390-a4a6-8cc49a664356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:50.260961 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ce0c1-cb8a-4390-a4a6-8cc49a664356 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:50.298852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 979/1938] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:50 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:50.302765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2069a-0afa-4f82-a476-8e8a07371b88 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:50.314638 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2069a-0afa-4f82-a476-8e8a07371b88 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:50.393207 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18f02b-6378-4054-a1f4-ae5299424d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:50.393970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18f02b-6378-4054-a1f4-ae5299424d8c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:50.394441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18f02b-6378-4054-a1f4-ae5299424d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:50.394814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18f02b-6378-4054-a1f4-ae5299424d8c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:50.395677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18f02b-6378-4054-a1f4-ae5299424d8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:50.401724 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18f02b-6378-4054-a1f4-ae5299424d8c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:50.414028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ce0c1-cb8a-4390-a4a6-8cc49a664356 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:50.415915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cd2ce0c1-cb8a-4390-a4a6-8cc49a664356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:50.431270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ce0c1-cb8a-4390-a4a6-8cc49a664356 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:50.452879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac5678-3da5-48e5-98f0-20774bd76449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:50.453681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac5678-3da5-48e5-98f0-20774bd76449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:50.454063 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac5678-3da5-48e5-98f0-20774bd76449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:50.454610 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac5678-3da5-48e5-98f0-20774bd76449 admin admin] PATH_INFO: `/v3/users/a251d538d8d949e080e691a11929b57b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:50.457143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac5678-3da5-48e5-98f0-20774bd76449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a251d538d8d949e080e691a11929b57b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:50.459298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac5678-3da5-48e5-98f0-20774bd76449 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:50.469787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 960/1939] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:50 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:50.471016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18f02b-6378-4054-a1f4-ae5299424d8c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:50.491380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18f02b-6378-4054-a1f4-ae5299424d8c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:50.498830 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4714665-8e8c-4569-84e6-9f31db548860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:50.499551 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4714665-8e8c-4569-84e6-9f31db548860 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:50.499844 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4714665-8e8c-4569-84e6-9f31db548860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:50.500121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4714665-8e8c-4569-84e6-9f31db548860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:50.500879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4714665-8e8c-4569-84e6-9f31db548860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:50.503292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4714665-8e8c-4569-84e6-9f31db548860 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:50.520788 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 961/1940] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:50.522258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4714665-8e8c-4569-84e6-9f31db548860 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:50.533137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e9ac5678-3da5-48e5-98f0-20774bd76449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a251d538d8d949e080e691a11929b57b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:50.543391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4714665-8e8c-4569-84e6-9f31db548860 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:50.546405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e9ac5678-3da5-48e5-98f0-20774bd76449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a251d538d8d949e080e691a11929b57b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:50.547548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e9ac5678-3da5-48e5-98f0-20774bd76449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a251d538d8d949e080e691a11929b57b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:50.552936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a297d2-91e6-45f3-ae75-1db1912de618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:50.553500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a297d2-91e6-45f3-ae75-1db1912de618 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:50.554931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a297d2-91e6-45f3-ae75-1db1912de618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:50.555240 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a297d2-91e6-45f3-ae75-1db1912de618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:50.556285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a297d2-91e6-45f3-ae75-1db1912de618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:50.565439 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a297d2-91e6-45f3-ae75-1db1912de618 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:50.570022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e9ac5678-3da5-48e5-98f0-20774bd76449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a251d538d8d949e080e691a11929b57b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:50.580266 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 962/1941] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:50.581901 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a297d2-91e6-45f3-ae75-1db1912de618 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:50.584004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-04a297d2-91e6-45f3-ae75-1db1912de618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:50.588854 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e9ac5678-3da5-48e5-98f0-20774bd76449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a251d538d8d949e080e691a11929b57b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:50.603587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a297d2-91e6-45f3-ae75-1db1912de618 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:50.605534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 980/1942] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:50 2022] DELETE /identity/v3/users/a251d538d8d949e080e691a11929b57b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:50.611091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b0792-5420-454f-8ebe-6afc4b6ea072 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:50.611091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b0792-5420-454f-8ebe-6afc4b6ea072 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:50.611397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b0792-5420-454f-8ebe-6afc4b6ea072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:50.640245 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-775b0792-5420-454f-8ebe-6afc4b6ea072 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:50.642105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-775b0792-5420-454f-8ebe-6afc4b6ea072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:50.695764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:50.721340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a59754-1ded-4636-9397-36a28a95a87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:50.722032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a59754-1ded-4636-9397-36a28a95a87d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:50.722312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a59754-1ded-4636-9397-36a28a95a87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:50.722639 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a59754-1ded-4636-9397-36a28a95a87d admin admin] PATH_INFO: `/v3/projects/7b9ec4dc507f4f249eb2fc644078c614/users/4cc888bab31844d6b5e737f500de30c9/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:50.755025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 981/1943] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:50.756919 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b0792-5420-454f-8ebe-6afc4b6ea072 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:50.759628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-775b0792-5420-454f-8ebe-6afc4b6ea072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:50.774945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a59754-1ded-4636-9397-36a28a95a87d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b9ec4dc507f4f249eb2fc644078c614, user_id=4cc888bab31844d6b5e737f500de30c9, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:50.778819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a59754-1ded-4636-9397-36a28a95a87d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:50.786325 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b0792-5420-454f-8ebe-6afc4b6ea072 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:50.798399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89d882-7ad4-41db-afbb-fdb02facdaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:50.799179 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89d882-7ad4-41db-afbb-fdb02facdaec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:50.799654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89d882-7ad4-41db-afbb-fdb02facdaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:50.799912 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89d882-7ad4-41db-afbb-fdb02facdaec admin admin] PATH_INFO: `/v3/projects/601b4970386147be930f13b9b590b223` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:50.801115 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89d882-7ad4-41db-afbb-fdb02facdaec admin admin] RBAC: Authorizing `identity:delete_project(project_id=601b4970386147be930f13b9b590b223)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:50.803067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 963/1944] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:50 2022] PUT /identity/v3/projects/7b9ec4dc507f4f249eb2fc644078c614/users/4cc888bab31844d6b5e737f500de30c9/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:50.804430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a59754-1ded-4636-9397-36a28a95a87d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:50.809698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89d882-7ad4-41db-afbb-fdb02facdaec admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:50.812350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a6a59754-1ded-4636-9397-36a28a95a87d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:50.878918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:50.904522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a59754-1ded-4636-9397-36a28a95a87d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:50.913483 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bf89d882-7ad4-41db-afbb-fdb02facdaec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '601b4970386147be930f13b9b590b223', 'request_id': 'req-bf89d882-7ad4-41db-afbb-fdb02facdaec'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:50.926891 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bf89d882-7ad4-41db-afbb-fdb02facdaec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '601b4970386147be930f13b9b590b223', 'request_id': 'req-bf89d882-7ad4-41db-afbb-fdb02facdaec'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:50.934435 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bf89d882-7ad4-41db-afbb-fdb02facdaec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '601b4970386147be930f13b9b590b223', 'request_id': 'req-bf89d882-7ad4-41db-afbb-fdb02facdaec'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:50.953695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bf89d882-7ad4-41db-afbb-fdb02facdaec admin admin] The token cache is being invalidate because project 601b4970386147be930f13b9b590b223 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:50.953994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bf89d882-7ad4-41db-afbb-fdb02facdaec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:50.955376 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bf89d882-7ad4-41db-afbb-fdb02facdaec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:50.959670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 982/1945] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:50 2022] DELETE /identity/v3/projects/601b4970386147be930f13b9b590b223 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:50.959670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb17349-9b24-417b-9e4a-17d3125685a8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:50.959670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb17349-9b24-417b-9e4a-17d3125685a8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:50.959670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb17349-9b24-417b-9e4a-17d3125685a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:50.977150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd55ab-6234-416c-963a-9d90de514b63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:50.977686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd55ab-6234-416c-963a-9d90de514b63 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:50.977965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd55ab-6234-416c-963a-9d90de514b63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:50.978236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd55ab-6234-416c-963a-9d90de514b63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:50.981743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-75fd55ab-6234-416c-963a-9d90de514b63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:50.995422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dfb17349-9b24-417b-9e4a-17d3125685a8 None None] MFA Rules not processed for user `4cc888bab31844d6b5e737f500de30c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:50.996731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dfb17349-9b24-417b-9e4a-17d3125685a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:51.081133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd55ab-6234-416c-963a-9d90de514b63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:51.082938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd55ab-6234-416c-963a-9d90de514b63 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:51.092198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 983/1946] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:51.095407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 964/1947] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:51.096349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb17349-9b24-417b-9e4a-17d3125685a8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:51.098014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd55ab-6234-416c-963a-9d90de514b63 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:51.098367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dfb17349-9b24-417b-9e4a-17d3125685a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:51.099452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-75fd55ab-6234-416c-963a-9d90de514b63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:51.113680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd55ab-6234-416c-963a-9d90de514b63 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:51.117069 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb17349-9b24-417b-9e4a-17d3125685a8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:51.120014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead21fb7-654f-480e-88be-0aa7d3c695ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:51.120967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead21fb7-654f-480e-88be-0aa7d3c695ca admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:51.121347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead21fb7-654f-480e-88be-0aa7d3c695ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:51.121805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead21fb7-654f-480e-88be-0aa7d3c695ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:51.121990 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:51.122900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:51.123262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead21fb7-654f-480e-88be-0aa7d3c695ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:51.123395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:51.123559 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] PATH_INFO: `/v3/users/c559fc41eb8a4409ba33d8e3ff9a2b8e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:51.124954 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c559fc41eb8a4409ba33d8e3ff9a2b8e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:51.126999 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead21fb7-654f-480e-88be-0aa7d3c695ca admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:51.129130 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:51.219125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 965/1948] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:51 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:51.227726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead21fb7-654f-480e-88be-0aa7d3c695ca admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:51.235295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c559fc41eb8a4409ba33d8e3ff9a2b8e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:51.255597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c559fc41eb8a4409ba33d8e3ff9a2b8e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:51.256231 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c559fc41eb8a4409ba33d8e3ff9a2b8e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:51.267253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:51.289406 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c559fc41eb8a4409ba33d8e3ff9a2b8e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:51.304185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c559fc41eb8a4409ba33d8e3ff9a2b8e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:51.317731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 984/1949] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:51 2022] DELETE /identity/v3/users/c559fc41eb8a4409ba33d8e3ff9a2b8e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:51.352596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead21fb7-654f-480e-88be-0aa7d3c695ca admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:51.384432 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:51.448448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc9502-9e20-4638-bd14-5c0666210a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:51.448785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc9502-9e20-4638-bd14-5c0666210a25 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:51.449587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc9502-9e20-4638-bd14-5c0666210a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:51.451031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc9502-9e20-4638-bd14-5c0666210a25 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:51.451742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc9502-9e20-4638-bd14-5c0666210a25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:51.453743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc9502-9e20-4638-bd14-5c0666210a25 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:51.494392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 966/1950] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:51 2022] POST /identity/v3/users => generated 289 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:51.499061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:51.500664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:51.517042 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd78342-3cb6-4102-b8a0-4e8a7fd3909c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:51.522155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ae770-47b5-47c2-bd37-e7df182c66f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:51.522783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ae770-47b5-47c2-bd37-e7df182c66f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:51.523077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ae770-47b5-47c2-bd37-e7df182c66f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:51.523369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ae770-47b5-47c2-bd37-e7df182c66f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:51.524026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ae770-47b5-47c2-bd37-e7df182c66f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:51.526218 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ae770-47b5-47c2-bd37-e7df182c66f6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:51.542285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 985/1951] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:51.543878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc9502-9e20-4638-bd14-5c0666210a25 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:51.555880 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc9502-9e20-4638-bd14-5c0666210a25 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:51.566839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f06744-20c9-4186-bfbf-07023e767076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:51.567754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f06744-20c9-4186-bfbf-07023e767076 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:51.568114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f06744-20c9-4186-bfbf-07023e767076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:51.568459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f06744-20c9-4186-bfbf-07023e767076 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:51.569236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f06744-20c9-4186-bfbf-07023e767076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:51.570935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f06744-20c9-4186-bfbf-07023e767076 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:51.588572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 967/1952] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:15:51.589783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ae770-47b5-47c2-bd37-e7df182c66f6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:51.611621 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ae770-47b5-47c2-bd37-e7df182c66f6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:51.625222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083007c1-ce69-4fa4-a748-0beeb0adc8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:51.626166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083007c1-ce69-4fa4-a748-0beeb0adc8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:51.626622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083007c1-ce69-4fa4-a748-0beeb0adc8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:51.627205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083007c1-ce69-4fa4-a748-0beeb0adc8b2 admin admin] PATH_INFO: `/v3/projects/f0440b0a20b2485bb4927b631a94db4e/users/e135acb5386940278b5972b0771d27b5/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:51.670144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083007c1-ce69-4fa4-a748-0beeb0adc8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0440b0a20b2485bb4927b631a94db4e, user_id=e135acb5386940278b5972b0771d27b5, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:51.676679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083007c1-ce69-4fa4-a748-0beeb0adc8b2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:51.709125 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 986/1953] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:51 2022] PUT /identity/v3/projects/f0440b0a20b2485bb4927b631a94db4e/users/e135acb5386940278b5972b0771d27b5/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:51.710281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ace90-63a9-4f26-9e4b-a599b4ffa43b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:51.710487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ace90-63a9-4f26-9e4b-a599b4ffa43b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:51.710949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ace90-63a9-4f26-9e4b-a599b4ffa43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:51.749767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ad4ace90-63a9-4f26-9e4b-a599b4ffa43b None None] MFA Rules not processed for user `e135acb5386940278b5972b0771d27b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:51.760013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ad4ace90-63a9-4f26-9e4b-a599b4ffa43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:51.858507 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 968/1954] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:51.863767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083007c1-ce69-4fa4-a748-0beeb0adc8b2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:51.964224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083007c1-ce69-4fa4-a748-0beeb0adc8b2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:51.970855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c4792-4e9d-4f6a-ad4a-2073733ee3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:51.971736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c4792-4e9d-4f6a-ad4a-2073733ee3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:51.972012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c4792-4e9d-4f6a-ad4a-2073733ee3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:51.972281 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c4792-4e9d-4f6a-ad4a-2073733ee3f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:51.972925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c4792-4e9d-4f6a-ad4a-2073733ee3f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:51.975096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c4792-4e9d-4f6a-ad4a-2073733ee3f6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:52.035229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 987/1955] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:51 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:52.038239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ace90-63a9-4f26-9e4b-a599b4ffa43b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:52.055352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ace90-63a9-4f26-9e4b-a599b4ffa43b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:52.106514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c4792-4e9d-4f6a-ad4a-2073733ee3f6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:52.108660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a55c4792-4e9d-4f6a-ad4a-2073733ee3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:52.130919 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebf624-7387-4be4-9536-ef68bdf17eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:52.131593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebf624-7387-4be4-9536-ef68bdf17eaa admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:52.131769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebf624-7387-4be4-9536-ef68bdf17eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:52.132041 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebf624-7387-4be4-9536-ef68bdf17eaa admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:52.132729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebf624-7387-4be4-9536-ef68bdf17eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:52.135443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebf624-7387-4be4-9536-ef68bdf17eaa admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:52.192003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 969/1956] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:52 2022] POST /identity/v3/users => generated 295 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:52.193333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebf624-7387-4be4-9536-ef68bdf17eaa admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:52.206730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebf624-7387-4be4-9536-ef68bdf17eaa admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:52.216503 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f27dfd-784a-4e71-a5fc-4c163690208d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:52.217946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f27dfd-784a-4e71-a5fc-4c163690208d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:52.218226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f27dfd-784a-4e71-a5fc-4c163690208d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:52.218499 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f27dfd-784a-4e71-a5fc-4c163690208d admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:52.219201 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f27dfd-784a-4e71-a5fc-4c163690208d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:52.221157 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f27dfd-784a-4e71-a5fc-4c163690208d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:52.237051 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 970/1957] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:52.241678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f27dfd-784a-4e71-a5fc-4c163690208d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:52.257796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c4792-4e9d-4f6a-ad4a-2073733ee3f6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:52.264283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f27dfd-784a-4e71-a5fc-4c163690208d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:52.271371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:52.272040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:52.272328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:52.272606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] PATH_INFO: `/v3/projects/6509bc9106d04912a51222396bd74741` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:52.274202 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6509bc9106d04912a51222396bd74741)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:52.281382 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:52.292447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9fdd4-330c-4783-a913-d8de59cc4658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:52.293152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9fdd4-330c-4783-a913-d8de59cc4658 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:52.293430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9fdd4-330c-4783-a913-d8de59cc4658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:52.293793 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9fdd4-330c-4783-a913-d8de59cc4658 admin admin] PATH_INFO: `/v3/projects/d40b9fc3085b4fabab28b9025a878dcb/users/3335536739a74a78aef8e4cbae9ff3c7/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:52.321240 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:52.325959 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9fdd4-330c-4783-a913-d8de59cc4658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d40b9fc3085b4fabab28b9025a878dcb, user_id=3335536739a74a78aef8e4cbae9ff3c7, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:52.329268 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9fdd4-330c-4783-a913-d8de59cc4658 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:52.360101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:52.382159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 971/1958] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:52 2022] PUT /identity/v3/projects/d40b9fc3085b4fabab28b9025a878dcb/users/3335536739a74a78aef8e4cbae9ff3c7/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:52.383487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9fdd4-330c-4783-a913-d8de59cc4658 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:52.390002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6509bc9106d04912a51222396bd74741', 'request_id': 'req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:52.415588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6509bc9106d04912a51222396bd74741', 'request_id': 'req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:52.434093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6509bc9106d04912a51222396bd74741', 'request_id': 'req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:52.474673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] The token cache is being invalidate because project 6509bc9106d04912a51222396bd74741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:52.475279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:52.476059 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:52.478263 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 988/1959] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:52 2022] DELETE /identity/v3/projects/6509bc9106d04912a51222396bd74741 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:52.479970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9fdd4-330c-4783-a913-d8de59cc4658 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:52.484247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:52.485830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:52.572194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055053e-d2ef-435f-af75-092bbf71b3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:52.579295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055053e-d2ef-435f-af75-092bbf71b3bb admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:52.580481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055053e-d2ef-435f-af75-092bbf71b3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:52.580481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055053e-d2ef-435f-af75-092bbf71b3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:52.582591 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055053e-d2ef-435f-af75-092bbf71b3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:52.585398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055053e-d2ef-435f-af75-092bbf71b3bb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:52.602325 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ccc09-9fca-4d76-8933-2c2e9c1f43bd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:52.610332 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 972/1960] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:52.619538 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429c14b-755c-463c-a1ce-471088ae69be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:52.620852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429c14b-755c-463c-a1ce-471088ae69be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:52.621139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429c14b-755c-463c-a1ce-471088ae69be admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:52.621490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429c14b-755c-463c-a1ce-471088ae69be admin admin] PATH_INFO: `/v3/users/dc3049ba43e34449a1ae9f00612181fe` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:52.622453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055053e-d2ef-435f-af75-092bbf71b3bb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:52.623851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429c14b-755c-463c-a1ce-471088ae69be admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc3049ba43e34449a1ae9f00612181fe)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:52.625814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429c14b-755c-463c-a1ce-471088ae69be admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:52.628469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9055053e-d2ef-435f-af75-092bbf71b3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:52.653249 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055053e-d2ef-435f-af75-092bbf71b3bb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:52.672029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbefa7-920b-4d16-8b4d-e5fd9f978de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:52.672755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbefa7-920b-4d16-8b4d-e5fd9f978de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:52.673106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbefa7-920b-4d16-8b4d-e5fd9f978de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:52.673565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbefa7-920b-4d16-8b4d-e5fd9f978de5 admin admin] PATH_INFO: `/v3/projects/d40b9fc3085b4fabab28b9025a878dcb/users/3335536739a74a78aef8e4cbae9ff3c7/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:52.677839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fbefa7-920b-4d16-8b4d-e5fd9f978de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d40b9fc3085b4fabab28b9025a878dcb, user_id=3335536739a74a78aef8e4cbae9ff3c7, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:52.681259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fbefa7-920b-4d16-8b4d-e5fd9f978de5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:52.695717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 973/1961] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:52 2022] PUT /identity/v3/projects/d40b9fc3085b4fabab28b9025a878dcb/users/3335536739a74a78aef8e4cbae9ff3c7/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:52.697086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbefa7-920b-4d16-8b4d-e5fd9f978de5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:52.699383 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c6fbefa7-920b-4d16-8b4d-e5fd9f978de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:52.731934 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e429c14b-755c-463c-a1ce-471088ae69be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc3049ba43e34449a1ae9f00612181fe'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:52.746650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e429c14b-755c-463c-a1ce-471088ae69be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc3049ba43e34449a1ae9f00612181fe'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:52.748385 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e429c14b-755c-463c-a1ce-471088ae69be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc3049ba43e34449a1ae9f00612181fe'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:52.772322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e429c14b-755c-463c-a1ce-471088ae69be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc3049ba43e34449a1ae9f00612181fe'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:52.785510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e429c14b-755c-463c-a1ce-471088ae69be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc3049ba43e34449a1ae9f00612181fe'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:52.803519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbefa7-920b-4d16-8b4d-e5fd9f978de5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:52.803858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 989/1962] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:52 2022] DELETE /identity/v3/users/dc3049ba43e34449a1ae9f00612181fe => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:52.806856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ab69a-7704-4c59-a152-2479a2b3c817 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:52.806856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ab69a-7704-4c59-a152-2479a2b3c817 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:52.806856 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ab69a-7704-4c59-a152-2479a2b3c817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:52.849699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4f9ab69a-7704-4c59-a152-2479a2b3c817 None None] MFA Rules not processed for user `3335536739a74a78aef8e4cbae9ff3c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:52.851082 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4f9ab69a-7704-4c59-a152-2479a2b3c817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:52.857796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:52.887665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:52.901207 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:52.903892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:52.904241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:52.904764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3 admin admin] PATH_INFO: `/v3/projects/46e086801d224545b1fd4cdc0c61e8a9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:52.906172 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46e086801d224545b1fd4cdc0c61e8a9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:52.912252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:52.977047 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 990/1963] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:52.985725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ab69a-7704-4c59-a152-2479a2b3c817 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:52.987687 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4f9ab69a-7704-4c59-a152-2479a2b3c817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:53.001139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e086801d224545b1fd4cdc0c61e8a9', 'request_id': 'req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:53.008201 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e086801d224545b1fd4cdc0c61e8a9', 'request_id': 'req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:53.016127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e086801d224545b1fd4cdc0c61e8a9', 'request_id': 'req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:53.039333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3 admin admin] The token cache is being invalidate because project 46e086801d224545b1fd4cdc0c61e8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:53.040064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:53.041171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-98fc3dba-6ceb-491d-b958-d4ca7a73d3b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:53.043187 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 974/1964] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:52 2022] DELETE /identity/v3/projects/46e086801d224545b1fd4cdc0c61e8a9 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:53.048626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f00859-2cb3-4ebb-b7ba-b5e86a5de039 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.049599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f00859-2cb3-4ebb-b7ba-b5e86a5de039 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.049599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f00859-2cb3-4ebb-b7ba-b5e86a5de039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.073474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ab69a-7704-4c59-a152-2479a2b3c817 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.082681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c6f00859-2cb3-4ebb-b7ba-b5e86a5de039 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:53.083940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c6f00859-2cb3-4ebb-b7ba-b5e86a5de039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:53.142478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63287e-00ea-4736-bc5e-b1581a00d2f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.143141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63287e-00ea-4736-bc5e-b1581a00d2f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.143421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63287e-00ea-4736-bc5e-b1581a00d2f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.143699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63287e-00ea-4736-bc5e-b1581a00d2f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.145423 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6e63287e-00ea-4736-bc5e-b1581a00d2f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:53.164188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 975/1965] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:53.172470 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef31a11-4f03-4fae-8d5e-3528834b2124 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.172758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef31a11-4f03-4fae-8d5e-3528834b2124 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.173038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef31a11-4f03-4fae-8d5e-3528834b2124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.193956 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bef31a11-4f03-4fae-8d5e-3528834b2124 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:53.195502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bef31a11-4f03-4fae-8d5e-3528834b2124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:53.207589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 976/1966] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:53.208700 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef31a11-4f03-4fae-8d5e-3528834b2124 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:53.218786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef31a11-4f03-4fae-8d5e-3528834b2124 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.220083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63287e-00ea-4736-bc5e-b1581a00d2f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.221779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63287e-00ea-4736-bc5e-b1581a00d2f0 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.223773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d80879-5f8e-49fe-9246-bfd89a885c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.224391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d80879-5f8e-49fe-9246-bfd89a885c76 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.224808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d80879-5f8e-49fe-9246-bfd89a885c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.225092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d80879-5f8e-49fe-9246-bfd89a885c76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.225748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d80879-5f8e-49fe-9246-bfd89a885c76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:15:53.226054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d80879-5f8e-49fe-9246-bfd89a885c76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.227884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d80879-5f8e-49fe-9246-bfd89a885c76 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.235023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 991/1967] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:15:53.246334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 977/1968] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:15:53.247195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63287e-00ea-4736-bc5e-b1581a00d2f0 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:53.248118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d80879-5f8e-49fe-9246-bfd89a885c76 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:53.249293 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6e63287e-00ea-4736-bc5e-b1581a00d2f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:53.261620 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d80879-5f8e-49fe-9246-bfd89a885c76 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.268379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6803d-da08-40a8-bc94-0f3e34176843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.268725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63287e-00ea-4736-bc5e-b1581a00d2f0 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.269495 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6803d-da08-40a8-bc94-0f3e34176843 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.269619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6803d-da08-40a8-bc94-0f3e34176843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.270103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6803d-da08-40a8-bc94-0f3e34176843 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.270758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6803d-da08-40a8-bc94-0f3e34176843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.273048 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6803d-da08-40a8-bc94-0f3e34176843 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.279945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.280889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.281441 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.281678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] PATH_INFO: `/v3/projects/2a060cc764074eff839aa7c74335b94a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.283000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a060cc764074eff839aa7c74335b94a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.285409 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.304011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 978/1969] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:53 2022] POST /identity/v3/projects => generated 409 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:53.304866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6803d-da08-40a8-bc94-0f3e34176843 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:53.306464 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-47c6803d-da08-40a8-bc94-0f3e34176843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:53.318124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:53.332294 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:53.347800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a060cc764074eff839aa7c74335b94a', 'request_id': 'req-851abf5e-a6b7-459e-9255-1fbd486a7663'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:53.358151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a060cc764074eff839aa7c74335b94a', 'request_id': 'req-851abf5e-a6b7-459e-9255-1fbd486a7663'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:53.379611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a060cc764074eff839aa7c74335b94a', 'request_id': 'req-851abf5e-a6b7-459e-9255-1fbd486a7663'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:53.389796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6803d-da08-40a8-bc94-0f3e34176843 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.392411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] The token cache is being invalidate because project 2a060cc764074eff839aa7c74335b94a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:53.392666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:53.393346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:53.395184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 992/1970] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:53 2022] DELETE /identity/v3/projects/2a060cc764074eff839aa7c74335b94a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:53.395862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:53.397184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:53.448654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f4a4b-a89a-42e3-87f5-bf63c6293150 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.449285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f4a4b-a89a-42e3-87f5-bf63c6293150 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.449590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f4a4b-a89a-42e3-87f5-bf63c6293150 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.450535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f4a4b-a89a-42e3-87f5-bf63c6293150 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.452665 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3b3f4a4b-a89a-42e3-87f5-bf63c6293150 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:53.471712 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851abf5e-a6b7-459e-9255-1fbd486a7663 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.476348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7ae42-31cf-408e-b763-073bb4318c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.476993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7ae42-31cf-408e-b763-073bb4318c48 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.477271 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7ae42-31cf-408e-b763-073bb4318c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.477540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7ae42-31cf-408e-b763-073bb4318c48 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.478172 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7ae42-31cf-408e-b763-073bb4318c48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.480142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7ae42-31cf-408e-b763-073bb4318c48 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.506276 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 993/1971] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:53 2022] POST /identity/v3/users => generated 305 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:53.510675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5124a-7fb1-4b7e-96d5-48875a1ca4fa None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.510954 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5124a-7fb1-4b7e-96d5-48875a1ca4fa None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.511252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5124a-7fb1-4b7e-96d5-48875a1ca4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.533009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f4a4b-a89a-42e3-87f5-bf63c6293150 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.534182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0be5124a-7fb1-4b7e-96d5-48875a1ca4fa None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:53.534581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f4a4b-a89a-42e3-87f5-bf63c6293150 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.535705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0be5124a-7fb1-4b7e-96d5-48875a1ca4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:53.545931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 979/1972] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:53.546826 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f4a4b-a89a-42e3-87f5-bf63c6293150 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:53.548978 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 994/1973] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:53.557116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962c96f-b3b7-491c-8c1c-19c1887d1fe6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.557401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962c96f-b3b7-491c-8c1c-19c1887d1fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.557673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962c96f-b3b7-491c-8c1c-19c1887d1fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.564762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f4a4b-a89a-42e3-87f5-bf63c6293150 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.570077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cba784-f92f-4e91-b430-085f81948969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.570921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cba784-f92f-4e91-b430-085f81948969 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.571312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cba784-f92f-4e91-b430-085f81948969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.571660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cba784-f92f-4e91-b430-085f81948969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.572616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cba784-f92f-4e91-b430-085f81948969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.574223 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cba784-f92f-4e91-b430-085f81948969 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.581428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7962c96f-b3b7-491c-8c1c-19c1887d1fe6 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:53.582819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7962c96f-b3b7-491c-8c1c-19c1887d1fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:53.585029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 980/1974] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:53.588888 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cba784-f92f-4e91-b430-085f81948969 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:53.600028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 995/1975] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:53.603659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cba784-f92f-4e91-b430-085f81948969 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.604518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962c96f-b3b7-491c-8c1c-19c1887d1fe6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:53.609402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36753fef-56bd-44c6-bba5-1de71cc97e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.610029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36753fef-56bd-44c6-bba5-1de71cc97e0b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.610313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36753fef-56bd-44c6-bba5-1de71cc97e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.610591 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36753fef-56bd-44c6-bba5-1de71cc97e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.611274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36753fef-56bd-44c6-bba5-1de71cc97e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.612887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36753fef-56bd-44c6-bba5-1de71cc97e0b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.625401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962c96f-b3b7-491c-8c1c-19c1887d1fe6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.626483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 981/1976] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:15:53.631365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7627172-7a32-473c-b2f0-fbaffc7ef4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.631653 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36753fef-56bd-44c6-bba5-1de71cc97e0b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:53.632191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7627172-7a32-473c-b2f0-fbaffc7ef4ad admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.632484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7627172-7a32-473c-b2f0-fbaffc7ef4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.632782 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7627172-7a32-473c-b2f0-fbaffc7ef4ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.633441 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7627172-7a32-473c-b2f0-fbaffc7ef4ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:15:53.633751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7627172-7a32-473c-b2f0-fbaffc7ef4ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.635373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7627172-7a32-473c-b2f0-fbaffc7ef4ad admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.642875 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36753fef-56bd-44c6-bba5-1de71cc97e0b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.647899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7cc91-ca61-4578-bfcf-952dc00acd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.648561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7cc91-ca61-4578-bfcf-952dc00acd96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.648837 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7cc91-ca61-4578-bfcf-952dc00acd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.649114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7cc91-ca61-4578-bfcf-952dc00acd96 admin admin] PATH_INFO: `/v3/projects/538f1079199a449b873d467088899c28/users/6c148ee64f8b43d2a7bf879c30bcb8ef/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.649691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 996/1977] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:15:53.654617 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7627172-7a32-473c-b2f0-fbaffc7ef4ad admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:53.665109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7627172-7a32-473c-b2f0-fbaffc7ef4ad admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.670641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76626fd-e586-4130-8099-ed5fde439d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.671277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76626fd-e586-4130-8099-ed5fde439d84 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.671650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76626fd-e586-4130-8099-ed5fde439d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.671914 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76626fd-e586-4130-8099-ed5fde439d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.672566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76626fd-e586-4130-8099-ed5fde439d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.672711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7cc91-ca61-4578-bfcf-952dc00acd96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538f1079199a449b873d467088899c28, user_id=6c148ee64f8b43d2a7bf879c30bcb8ef, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.674506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76626fd-e586-4130-8099-ed5fde439d84 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.675876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7cc91-ca61-4578-bfcf-952dc00acd96 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.694355 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 982/1978] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:53 2022] PUT /identity/v3/projects/538f1079199a449b873d467088899c28/users/6c148ee64f8b43d2a7bf879c30bcb8ef/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:53.703283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 997/1979] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:53 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:53.706934 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7cc91-ca61-4578-bfcf-952dc00acd96 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:53.707524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926c203-5e87-4a1c-8254-7791cbcf0384 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.707878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926c203-5e87-4a1c-8254-7791cbcf0384 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.708227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926c203-5e87-4a1c-8254-7791cbcf0384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.745324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c926c203-5e87-4a1c-8254-7791cbcf0384 None None] MFA Rules not processed for user `6c148ee64f8b43d2a7bf879c30bcb8ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:53.747402 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c926c203-5e87-4a1c-8254-7791cbcf0384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:53.760356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:53.784233 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7cc91-ca61-4578-bfcf-952dc00acd96 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.799490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:53.834084 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 998/1980] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:53 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:53.838685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926c203-5e87-4a1c-8254-7791cbcf0384 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:53.856801 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41902096-8d74-473b-8ce5-12580c277fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.857616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41902096-8d74-473b-8ce5-12580c277fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.858111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41902096-8d74-473b-8ce5-12580c277fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.858538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41902096-8d74-473b-8ce5-12580c277fa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.859894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41902096-8d74-473b-8ce5-12580c277fa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.862762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41902096-8d74-473b-8ce5-12580c277fa0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.890425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 983/1981] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:53 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:15:53.895234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41902096-8d74-473b-8ce5-12580c277fa0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:53.921413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41902096-8d74-473b-8ce5-12580c277fa0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.927550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ad2a0-c0d7-4dc5-a385-5f6b446fad69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.928424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ad2a0-c0d7-4dc5-a385-5f6b446fad69 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.928838 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ad2a0-c0d7-4dc5-a385-5f6b446fad69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.930747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926c203-5e87-4a1c-8254-7791cbcf0384 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:53.937900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1c87d-d15b-44bd-9cf7-fcef48165f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:53.938725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1c87d-d15b-44bd-9cf7-fcef48165f15 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:53.939157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1c87d-d15b-44bd-9cf7-fcef48165f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:53.939447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ad2a0-c0d7-4dc5-a385-5f6b446fad69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.939779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1c87d-d15b-44bd-9cf7-fcef48165f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:53.940575 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1c87d-d15b-44bd-9cf7-fcef48165f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.942032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ad2a0-c0d7-4dc5-a385-5f6b446fad69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:53.944107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1c87d-d15b-44bd-9cf7-fcef48165f15 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.948510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ad2a0-c0d7-4dc5-a385-5f6b446fad69 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:53.976331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 984/1982] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:53.978433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ad2a0-c0d7-4dc5-a385-5f6b446fad69 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:54.001818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 999/1983] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:53 2022] POST /identity/v3/projects => generated 407 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:54.001818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1c87d-d15b-44bd-9cf7-fcef48165f15 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:54.002742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ad2a0-c0d7-4dc5-a385-5f6b446fad69 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:54.114791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1c87d-d15b-44bd-9cf7-fcef48165f15 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:54.117887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0455e8b-42da-407a-b4b4-74420e98abc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:54.118508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0455e8b-42da-407a-b4b4-74420e98abc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:54.118925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0455e8b-42da-407a-b4b4-74420e98abc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:54.119236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0455e8b-42da-407a-b4b4-74420e98abc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:54.119885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0455e8b-42da-407a-b4b4-74420e98abc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:54.121486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0455e8b-42da-407a-b4b4-74420e98abc8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:54.123889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8265e5-ace4-4a29-a69e-d4d91b2401db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:54.125014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8265e5-ace4-4a29-a69e-d4d91b2401db admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:54.125489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8265e5-ace4-4a29-a69e-d4d91b2401db admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:54.126004 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8265e5-ace4-4a29-a69e-d4d91b2401db admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:54.126844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8265e5-ace4-4a29-a69e-d4d91b2401db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:54.129162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8265e5-ace4-4a29-a69e-d4d91b2401db admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:54.133761 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 985/1984] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:54.139222 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0455e8b-42da-407a-b4b4-74420e98abc8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:54.153544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0455e8b-42da-407a-b4b4-74420e98abc8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:54.156237 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1000/1985] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:53 2022] POST /identity/v3/users => generated 310 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:54.159898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8265e5-ace4-4a29-a69e-d4d91b2401db admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:54.161228 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03105645-3139-4ef1-8395-92ae9fa414cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:54.161917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03105645-3139-4ef1-8395-92ae9fa414cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:54.162199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03105645-3139-4ef1-8395-92ae9fa414cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:54.162476 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03105645-3139-4ef1-8395-92ae9fa414cd admin admin] PATH_INFO: `/v3/projects/94c9131322c74cdeb5ae7a514bec97db/users/d9431ce2c3964752a43047b5d51b57a0/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:54.172766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8265e5-ace4-4a29-a69e-d4d91b2401db admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:54.178706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd5632-c902-4680-b7cd-78360dba3c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:54.179803 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd5632-c902-4680-b7cd-78360dba3c36 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:54.180327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03105645-3139-4ef1-8395-92ae9fa414cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=94c9131322c74cdeb5ae7a514bec97db, user_id=d9431ce2c3964752a43047b5d51b57a0, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:54.181330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd5632-c902-4680-b7cd-78360dba3c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:54.181936 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd5632-c902-4680-b7cd-78360dba3c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:54.182764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd5632-c902-4680-b7cd-78360dba3c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:54.183085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03105645-3139-4ef1-8395-92ae9fa414cd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:54.185531 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd5632-c902-4680-b7cd-78360dba3c36 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:54.201294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1001/1986] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:54.203310 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 986/1987] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:54 2022] PUT /identity/v3/projects/94c9131322c74cdeb5ae7a514bec97db/users/d9431ce2c3964752a43047b5d51b57a0/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:54.203429 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd5632-c902-4680-b7cd-78360dba3c36 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:54.214608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa510a3-60d9-433a-98fc-cf8f9bcb5494 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:54.214917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa510a3-60d9-433a-98fc-cf8f9bcb5494 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:54.215225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa510a3-60d9-433a-98fc-cf8f9bcb5494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:54.221779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd5632-c902-4680-b7cd-78360dba3c36 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:54.249131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ffa510a3-60d9-433a-98fc-cf8f9bcb5494 None None] MFA Rules not processed for user `d9431ce2c3964752a43047b5d51b57a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:54.250746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ffa510a3-60d9-433a-98fc-cf8f9bcb5494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:54.272295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:54.282604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:54.303112 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4ebb1-6c7b-4981-9df2-0e9c5992009a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:54.303751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4ebb1-6c7b-4981-9df2-0e9c5992009a admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:54.304168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4ebb1-6c7b-4981-9df2-0e9c5992009a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:54.304539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4ebb1-6c7b-4981-9df2-0e9c5992009a admin admin] PATH_INFO: `/v3/projects/6030e0bb28f44f2ca585f254fdb5bcac/users/619f17dc752e4363a726a6b6121bffbb/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:54.334573 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 987/1988] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:15:54.340262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4ebb1-6c7b-4981-9df2-0e9c5992009a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6030e0bb28f44f2ca585f254fdb5bcac, user_id=619f17dc752e4363a726a6b6121bffbb, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:54.344014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4ebb1-6c7b-4981-9df2-0e9c5992009a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:54.348011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa510a3-60d9-433a-98fc-cf8f9bcb5494 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:54.350477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ffa510a3-60d9-433a-98fc-cf8f9bcb5494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:54.363437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1002/1989] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:54 2022] PUT /identity/v3/projects/6030e0bb28f44f2ca585f254fdb5bcac/users/619f17dc752e4363a726a6b6121bffbb/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:54.368863 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4ebb1-6c7b-4981-9df2-0e9c5992009a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:54.456104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4ebb1-6c7b-4981-9df2-0e9c5992009a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:54.460713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa510a3-60d9-433a-98fc-cf8f9bcb5494 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:54.461763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d06abb-9a29-4ef9-a505-ec960b0f02ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:54.462392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d06abb-9a29-4ef9-a505-ec960b0f02ba admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:54.462680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d06abb-9a29-4ef9-a505-ec960b0f02ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:54.463044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d06abb-9a29-4ef9-a505-ec960b0f02ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:54.463701 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d06abb-9a29-4ef9-a505-ec960b0f02ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:54.465951 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d06abb-9a29-4ef9-a505-ec960b0f02ba admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:54.497529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1003/1990] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:15:54.499079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d06abb-9a29-4ef9-a505-ec960b0f02ba admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:54.513072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d06abb-9a29-4ef9-a505-ec960b0f02ba admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:54.519581 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4021f-8f8d-4000-93de-86aff07369db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:54.520339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4021f-8f8d-4000-93de-86aff07369db admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:54.521066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4021f-8f8d-4000-93de-86aff07369db admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:54.522130 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4021f-8f8d-4000-93de-86aff07369db admin admin] PATH_INFO: `/v3/projects/6030e0bb28f44f2ca585f254fdb5bcac/users/619f17dc752e4363a726a6b6121bffbb/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:54.524935 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4021f-8f8d-4000-93de-86aff07369db admin admin] RBAC: Authorizing `identity:create_grant(project_id=6030e0bb28f44f2ca585f254fdb5bcac, user_id=619f17dc752e4363a726a6b6121bffbb, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:54.526589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081367a-8fba-4090-8c16-3c69e3010198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:54.527336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081367a-8fba-4090-8c16-3c69e3010198 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:54.527612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081367a-8fba-4090-8c16-3c69e3010198 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:54.528398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081367a-8fba-4090-8c16-3c69e3010198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:54.529992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4021f-8f8d-4000-93de-86aff07369db admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:54.546296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1004/1991] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:54 2022] PUT /identity/v3/projects/6030e0bb28f44f2ca585f254fdb5bcac/users/619f17dc752e4363a726a6b6121bffbb/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:54.557032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc588b6d-ff4d-4237-a1c4-cdeb0f6216b1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:54.560472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc588b6d-ff4d-4237-a1c4-cdeb0f6216b1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:54.561351 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc588b6d-ff4d-4237-a1c4-cdeb0f6216b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:54.600009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-cc588b6d-ff4d-4237-a1c4-cdeb0f6216b1 None None] MFA Rules not processed for user `619f17dc752e4363a726a6b6121bffbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:54.601799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cc588b6d-ff4d-4237-a1c4-cdeb0f6216b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:54.608250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3081367a-8fba-4090-8c16-3c69e3010198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:54.610042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3081367a-8fba-4090-8c16-3c69e3010198 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:54.634032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 988/1992] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:54.696036 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1005/1993] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:54 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:54.708363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081367a-8fba-4090-8c16-3c69e3010198 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:54.709823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3081367a-8fba-4090-8c16-3c69e3010198 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:54.722306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081367a-8fba-4090-8c16-3c69e3010198 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:54.769908 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:54.787170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6008c-789a-4f78-b657-87218e6dfd1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:54.787796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6008c-789a-4f78-b657-87218e6dfd1b service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:54.788072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6008c-789a-4f78-b657-87218e6dfd1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:54.788341 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6008c-789a-4f78-b657-87218e6dfd1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:54.801201 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6008c-789a-4f78-b657-87218e6dfd1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:54.803371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6008c-789a-4f78-b657-87218e6dfd1b service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:54.815409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 989/1994] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:54 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:55.010015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc588b6d-ff4d-4237-a1c4-cdeb0f6216b1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:55.021223 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc588b6d-ff4d-4237-a1c4-cdeb0f6216b1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:55.025351 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2454f-70ea-4930-8ae9-9a5e53dc3595 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:55.026157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2454f-70ea-4930-8ae9-9a5e53dc3595 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:55.026468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2454f-70ea-4930-8ae9-9a5e53dc3595 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:55.026841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2454f-70ea-4930-8ae9-9a5e53dc3595 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:55.064161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:55.089320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2454f-70ea-4930-8ae9-9a5e53dc3595 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:55.091729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2454f-70ea-4930-8ae9-9a5e53dc3595 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:55.104873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1006/1995] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:55 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:56.657465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6008c-789a-4f78-b657-87218e6dfd1b service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:56.659790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fef6008c-789a-4f78-b657-87218e6dfd1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:56.756803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6008c-789a-4f78-b657-87218e6dfd1b service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:56.767258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:56.767909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:56.768188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:56.768462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] PATH_INFO: `/v3/users/4cc888bab31844d6b5e737f500de30c9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:56.769617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc888bab31844d6b5e737f500de30c9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:56.771611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:56.819257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc888bab31844d6b5e737f500de30c9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:56.827125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc888bab31844d6b5e737f500de30c9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:56.839928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc888bab31844d6b5e737f500de30c9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:56.840660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc888bab31844d6b5e737f500de30c9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:56.847789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc888bab31844d6b5e737f500de30c9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:56.853792 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 990/1996] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:56 2022] DELETE /identity/v3/users/4cc888bab31844d6b5e737f500de30c9 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:56.859100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176b399-cbf7-4a24-af6b-8dec74de2825 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:56.859626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176b399-cbf7-4a24-af6b-8dec74de2825 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:56.860016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176b399-cbf7-4a24-af6b-8dec74de2825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:56.883574 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4176b399-cbf7-4a24-af6b-8dec74de2825 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:56.884771 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4176b399-cbf7-4a24-af6b-8dec74de2825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:56.963665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1007/1997] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:56.973792 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:56.975373 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:57.046170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995f064-2a44-4705-9f79-61fce1b61236 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:57.050937 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cee49c-44f2-4edf-a545-b39bea3b1f5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:57.051605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cee49c-44f2-4edf-a545-b39bea3b1f5c service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:57.051886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cee49c-44f2-4edf-a545-b39bea3b1f5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:57.052160 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cee49c-44f2-4edf-a545-b39bea3b1f5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:57.064660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cee49c-44f2-4edf-a545-b39bea3b1f5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:57.066801 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cee49c-44f2-4edf-a545-b39bea3b1f5c service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:57.078797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 991/1998] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:57.385916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176b399-cbf7-4a24-af6b-8dec74de2825 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:57.388167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4176b399-cbf7-4a24-af6b-8dec74de2825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:57.400375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176b399-cbf7-4a24-af6b-8dec74de2825 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:57.405462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:57.406108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:57.406387 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:57.406661 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] PATH_INFO: `/v3/projects/7b9ec4dc507f4f249eb2fc644078c614` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:57.407830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b9ec4dc507f4f249eb2fc644078c614)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:57.409762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:57.458535 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b9ec4dc507f4f249eb2fc644078c614', 'request_id': 'req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:57.466283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b9ec4dc507f4f249eb2fc644078c614', 'request_id': 'req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:57.472194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b9ec4dc507f4f249eb2fc644078c614', 'request_id': 'req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:57.483685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] The token cache is being invalidate because project 7b9ec4dc507f4f249eb2fc644078c614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:57.483975 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:57.484648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:57.486161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1008/1999] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:57 2022] DELETE /identity/v3/projects/7b9ec4dc507f4f249eb2fc644078c614 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:15:57.490686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cee49c-44f2-4edf-a545-b39bea3b1f5c service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:57.499730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:57.504352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cee49c-44f2-4edf-a545-b39bea3b1f5c service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:57.563492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:57.564145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:57.564420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:57.564691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] PATH_INFO: `/v3/users/e135acb5386940278b5972b0771d27b5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:57.565798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e135acb5386940278b5972b0771d27b5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:57.567797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:57.615969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e135acb5386940278b5972b0771d27b5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:57.623567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e135acb5386940278b5972b0771d27b5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:57.635591 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e135acb5386940278b5972b0771d27b5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:57.636279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e135acb5386940278b5972b0771d27b5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:57.643252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e135acb5386940278b5972b0771d27b5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:57.649227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 992/2000] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:57 2022] DELETE /identity/v3/users/e135acb5386940278b5972b0771d27b5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:57.929356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:57.930895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:57.945069 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b3c2d-d14b-4325-9fca-2ed22455e3a6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:58.001217 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:58.008304 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:58.009089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:58.009665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:58.010011 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] PATH_INFO: `/v3/projects/f0440b0a20b2485bb4927b631a94db4e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:58.011290 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0440b0a20b2485bb4927b631a94db4e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:58.013732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:58.064268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0440b0a20b2485bb4927b631a94db4e', 'request_id': 'req-a03334f1-8bf1-4a01-b553-9bb6df6f197c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.071651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0440b0a20b2485bb4927b631a94db4e', 'request_id': 'req-a03334f1-8bf1-4a01-b553-9bb6df6f197c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.077658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0440b0a20b2485bb4927b631a94db4e', 'request_id': 'req-a03334f1-8bf1-4a01-b553-9bb6df6f197c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.088639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] The token cache is being invalidate because project f0440b0a20b2485bb4927b631a94db4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:58.088923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.089547 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.091915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1009/2001] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:57 2022] DELETE /identity/v3/projects/f0440b0a20b2485bb4927b631a94db4e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:58.096271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:58.097992 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:58.112254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ee936-802d-4b7e-8813-fcc37172b1f0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:58.170000 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50ae99-879c-41a5-8281-5919adc0a451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:58.170655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50ae99-879c-41a5-8281-5919adc0a451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:58.170930 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50ae99-879c-41a5-8281-5919adc0a451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:58.171237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50ae99-879c-41a5-8281-5919adc0a451 admin admin] PATH_INFO: `/v3/users/3335536739a74a78aef8e4cbae9ff3c7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:58.172531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50ae99-879c-41a5-8281-5919adc0a451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3335536739a74a78aef8e4cbae9ff3c7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:58.175187 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50ae99-879c-41a5-8281-5919adc0a451 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:58.209299 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:58.233018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5d50ae99-879c-41a5-8281-5919adc0a451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3335536739a74a78aef8e4cbae9ff3c7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.240753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5d50ae99-879c-41a5-8281-5919adc0a451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3335536739a74a78aef8e4cbae9ff3c7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.256698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5d50ae99-879c-41a5-8281-5919adc0a451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3335536739a74a78aef8e4cbae9ff3c7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.257380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5d50ae99-879c-41a5-8281-5919adc0a451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3335536739a74a78aef8e4cbae9ff3c7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.265242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5d50ae99-879c-41a5-8281-5919adc0a451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3335536739a74a78aef8e4cbae9ff3c7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.272284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 993/2002] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:15:58 2022] DELETE /identity/v3/users/3335536739a74a78aef8e4cbae9ff3c7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:58.550113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:58.551674 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:58.564986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03334f1-8bf1-4a01-b553-9bb6df6f197c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:58.625905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275b888-08b1-4d40-bbde-efa4a670a83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:58.626581 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275b888-08b1-4d40-bbde-efa4a670a83b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:58.626861 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275b888-08b1-4d40-bbde-efa4a670a83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:58.627164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275b888-08b1-4d40-bbde-efa4a670a83b admin admin] PATH_INFO: `/v3/projects/d40b9fc3085b4fabab28b9025a878dcb` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:58.628287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275b888-08b1-4d40-bbde-efa4a670a83b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d40b9fc3085b4fabab28b9025a878dcb)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:58.630232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275b888-08b1-4d40-bbde-efa4a670a83b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:58.677103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9275b888-08b1-4d40-bbde-efa4a670a83b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd40b9fc3085b4fabab28b9025a878dcb', 'request_id': 'req-9275b888-08b1-4d40-bbde-efa4a670a83b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.684749 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9275b888-08b1-4d40-bbde-efa4a670a83b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd40b9fc3085b4fabab28b9025a878dcb', 'request_id': 'req-9275b888-08b1-4d40-bbde-efa4a670a83b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.690526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9275b888-08b1-4d40-bbde-efa4a670a83b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd40b9fc3085b4fabab28b9025a878dcb', 'request_id': 'req-9275b888-08b1-4d40-bbde-efa4a670a83b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.702314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9275b888-08b1-4d40-bbde-efa4a670a83b admin admin] The token cache is being invalidate because project d40b9fc3085b4fabab28b9025a878dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:15:58.702598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9275b888-08b1-4d40-bbde-efa4a670a83b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.703261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9275b888-08b1-4d40-bbde-efa4a670a83b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:15:58.704786 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1010/2003] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:15:58 2022] DELETE /identity/v3/projects/d40b9fc3085b4fabab28b9025a878dcb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:58.711139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0fee1-f21b-4650-9320-fa01a36f99a2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:58.711432 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0fee1-f21b-4650-9320-fa01a36f99a2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:58.711712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0fee1-f21b-4650-9320-fa01a36f99a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:58.732798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-05f0fee1-f21b-4650-9320-fa01a36f99a2 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:58.734262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-05f0fee1-f21b-4650-9320-fa01a36f99a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:58.811697 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 994/2004] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:58.820959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5816d-e62d-4d7f-af26-c04fd21456a0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:58.821383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5816d-e62d-4d7f-af26-c04fd21456a0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:58.821765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5816d-e62d-4d7f-af26-c04fd21456a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:58.845662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-46a5816d-e62d-4d7f-af26-c04fd21456a0 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:58.847034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-46a5816d-e62d-4d7f-af26-c04fd21456a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:58.855916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1011/2005] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:58.861319 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0fee1-f21b-4650-9320-fa01a36f99a2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:58.871332 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0fee1-f21b-4650-9320-fa01a36f99a2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:58.876077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8a6f3-f6d0-4c57-9ecb-e7cadb542fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:58.876681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a6f3-f6d0-4c57-9ecb-e7cadb542fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:58.876963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a6f3-f6d0-4c57-9ecb-e7cadb542fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:58.877238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a6f3-f6d0-4c57-9ecb-e7cadb542fa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:58.877882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8a6f3-f6d0-4c57-9ecb-e7cadb542fa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:15:58.878188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8a6f3-f6d0-4c57-9ecb-e7cadb542fa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:58.879826 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8a6f3-f6d0-4c57-9ecb-e7cadb542fa5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:58.891792 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 995/2006] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:15:58.896253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5816d-e62d-4d7f-af26-c04fd21456a0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:58.906279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5816d-e62d-4d7f-af26-c04fd21456a0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:58.911007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85a6fa-f2b1-4fa0-b467-1b2fb07bf507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:58.911642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85a6fa-f2b1-4fa0-b467-1b2fb07bf507 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:58.911928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85a6fa-f2b1-4fa0-b467-1b2fb07bf507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:58.912205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85a6fa-f2b1-4fa0-b467-1b2fb07bf507 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:58.912855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85a6fa-f2b1-4fa0-b467-1b2fb07bf507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:58.914766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85a6fa-f2b1-4fa0-b467-1b2fb07bf507 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:58.925105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:58.948502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1012/2007] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:15:58 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:58.949630 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8a6f3-f6d0-4c57-9ecb-e7cadb542fa5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:58.960535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8a6f3-f6d0-4c57-9ecb-e7cadb542fa5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:59.021073 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee3fb2-b478-4950-b557-5dfaae60414c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:59.021692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee3fb2-b478-4950-b557-5dfaae60414c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:59.021969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee3fb2-b478-4950-b557-5dfaae60414c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:59.022234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee3fb2-b478-4950-b557-5dfaae60414c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:59.022862 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee3fb2-b478-4950-b557-5dfaae60414c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:59.024855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee3fb2-b478-4950-b557-5dfaae60414c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:59.050754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 996/2008] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:15:58 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:15:59.052252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85a6fa-f2b1-4fa0-b467-1b2fb07bf507 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:59.063864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85a6fa-f2b1-4fa0-b467-1b2fb07bf507 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:59.068880 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a886a-c520-4d52-8d05-863976c1d700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:59.069518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a886a-c520-4d52-8d05-863976c1d700 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:59.069794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a886a-c520-4d52-8d05-863976c1d700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:59.070209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a886a-c520-4d52-8d05-863976c1d700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:59.070878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a886a-c520-4d52-8d05-863976c1d700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:59.072485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a886a-c520-4d52-8d05-863976c1d700 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:59.083744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1013/2009] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:59.087947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee3fb2-b478-4950-b557-5dfaae60414c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:59.098221 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee3fb2-b478-4950-b557-5dfaae60414c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:59.103393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3a9ac-4c27-454f-896d-7f237c50a218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:59.104003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3a9ac-4c27-454f-896d-7f237c50a218 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:59.104282 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3a9ac-4c27-454f-896d-7f237c50a218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:59.104559 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3a9ac-4c27-454f-896d-7f237c50a218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:59.105233 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3a9ac-4c27-454f-896d-7f237c50a218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:59.110798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3a9ac-4c27-454f-896d-7f237c50a218 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:59.118184 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 997/2010] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:15:59.122888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a886a-c520-4d52-8d05-863976c1d700 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:59.132796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a886a-c520-4d52-8d05-863976c1d700 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:59.137561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931fd95-ba0c-48ad-a9b8-2b0fd953ab86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:59.138214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931fd95-ba0c-48ad-a9b8-2b0fd953ab86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:59.138496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931fd95-ba0c-48ad-a9b8-2b0fd953ab86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:59.138771 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931fd95-ba0c-48ad-a9b8-2b0fd953ab86 admin admin] PATH_INFO: `/v3/projects/eb531234364c4f94a994e58c2b861199/users/1eb9b5b3061e4330b19882b62cc4e5f2/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:59.153795 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931fd95-ba0c-48ad-a9b8-2b0fd953ab86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb531234364c4f94a994e58c2b861199, user_id=1eb9b5b3061e4330b19882b62cc4e5f2, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:59.156490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931fd95-ba0c-48ad-a9b8-2b0fd953ab86 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:59.170138 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1014/2011] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:15:59 2022] PUT /identity/v3/projects/eb531234364c4f94a994e58c2b861199/users/1eb9b5b3061e4330b19882b62cc4e5f2/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:15:59.175221 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19a471-b90b-4f9a-bf9d-3f7f1fb2e59d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:59.175532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19a471-b90b-4f9a-bf9d-3f7f1fb2e59d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:59.175811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19a471-b90b-4f9a-bf9d-3f7f1fb2e59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:59.194921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ef19a471-b90b-4f9a-bf9d-3f7f1fb2e59d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:15:59.196166 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ef19a471-b90b-4f9a-bf9d-3f7f1fb2e59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:59.200765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:59.257698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 998/2012] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:15:59.268068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931fd95-ba0c-48ad-a9b8-2b0fd953ab86 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:59.269773 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3931fd95-ba0c-48ad-a9b8-2b0fd953ab86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:59.330103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931fd95-ba0c-48ad-a9b8-2b0fd953ab86 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:59.333760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9503e3-ca2e-4c8e-a2ca-2bd7860c5124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:59.334369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9503e3-ca2e-4c8e-a2ca-2bd7860c5124 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:59.334646 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9503e3-ca2e-4c8e-a2ca-2bd7860c5124 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:59.334919 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9503e3-ca2e-4c8e-a2ca-2bd7860c5124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:59.344399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9503e3-ca2e-4c8e-a2ca-2bd7860c5124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:59.345819 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9503e3-ca2e-4c8e-a2ca-2bd7860c5124 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:59.355075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1015/2013] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:00.104032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9c310-63ca-4457-a2f6-4804119833d6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:00.104848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9c310-63ca-4457-a2f6-4804119833d6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:00.105128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9c310-63ca-4457-a2f6-4804119833d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:00.140328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-afe9c310-63ca-4457-a2f6-4804119833d6 None None] MFA Rules not processed for user `1eb9b5b3061e4330b19882b62cc4e5f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:00.141554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-afe9c310-63ca-4457-a2f6-4804119833d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:00.224151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 999/2014] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:00.228800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9503e3-ca2e-4c8e-a2ca-2bd7860c5124 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:00.239937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9503e3-ca2e-4c8e-a2ca-2bd7860c5124 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:00.244866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb45187-f2de-438f-a827-82e6c1eeaa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:00.245575 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb45187-f2de-438f-a827-82e6c1eeaa5d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:00.245922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb45187-f2de-438f-a827-82e6c1eeaa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:00.246272 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb45187-f2de-438f-a827-82e6c1eeaa5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:00.246982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb45187-f2de-438f-a827-82e6c1eeaa5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:00.248942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb45187-f2de-438f-a827-82e6c1eeaa5d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:00.283579 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1016/2015] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:00 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:00.287239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9c310-63ca-4457-a2f6-4804119833d6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:00.299312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9c310-63ca-4457-a2f6-4804119833d6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:00.360616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f0647-cb64-4538-8422-b8f836340e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:00.361269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f0647-cb64-4538-8422-b8f836340e83 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:00.361551 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f0647-cb64-4538-8422-b8f836340e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:00.361823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f0647-cb64-4538-8422-b8f836340e83 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:00.362455 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f0647-cb64-4538-8422-b8f836340e83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:00.364709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f0647-cb64-4538-8422-b8f836340e83 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:00.390641 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1000/2016] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:00 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:00.391851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb45187-f2de-438f-a827-82e6c1eeaa5d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:00.402582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb45187-f2de-438f-a827-82e6c1eeaa5d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:00.407968 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be40ab-e4d0-426c-9461-38566139ef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:00.408702 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be40ab-e4d0-426c-9461-38566139ef65 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:00.408842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be40ab-e4d0-426c-9461-38566139ef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:00.409114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be40ab-e4d0-426c-9461-38566139ef65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:00.409756 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be40ab-e4d0-426c-9461-38566139ef65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:00.416104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be40ab-e4d0-426c-9461-38566139ef65 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:00.427219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1017/2017] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:00.431424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f0647-cb64-4538-8422-b8f836340e83 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:00.443552 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f0647-cb64-4538-8422-b8f836340e83 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:00.448671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4a6fc-1199-4fd9-ba4c-b129bc6f25aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:00.449357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4a6fc-1199-4fd9-ba4c-b129bc6f25aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:00.449776 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4a6fc-1199-4fd9-ba4c-b129bc6f25aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:00.449993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4a6fc-1199-4fd9-ba4c-b129bc6f25aa admin admin] PATH_INFO: `/v3/projects/0a88369caf124bf9ad9b7edf3e145009/users/7cbaeeff37314ec7bfb8cd8e26243e30/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:00.466590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4a6fc-1199-4fd9-ba4c-b129bc6f25aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a88369caf124bf9ad9b7edf3e145009, user_id=7cbaeeff37314ec7bfb8cd8e26243e30, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:00.469336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4a6fc-1199-4fd9-ba4c-b129bc6f25aa admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:00.485336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1001/2018] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:00 2022] PUT /identity/v3/projects/0a88369caf124bf9ad9b7edf3e145009/users/7cbaeeff37314ec7bfb8cd8e26243e30/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:00.490061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be40ab-e4d0-426c-9461-38566139ef65 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:00.501533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be40ab-e4d0-426c-9461-38566139ef65 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:00.566887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3af91-0ade-4ad2-badf-c9ff3d06e724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:00.567746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3af91-0ade-4ad2-badf-c9ff3d06e724 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:00.568219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3af91-0ade-4ad2-badf-c9ff3d06e724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:00.568563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3af91-0ade-4ad2-badf-c9ff3d06e724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:00.569645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3af91-0ade-4ad2-badf-c9ff3d06e724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:00.572019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3af91-0ade-4ad2-badf-c9ff3d06e724 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:00.584218 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1018/2019] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:00.588582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4a6fc-1199-4fd9-ba4c-b129bc6f25aa admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:00.599131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:00.603674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4a6fc-1199-4fd9-ba4c-b129bc6f25aa admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:00.617719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793bf66-d23b-4e79-a4cb-271ee6793414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:00.618486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793bf66-d23b-4e79-a4cb-271ee6793414 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:00.618851 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793bf66-d23b-4e79-a4cb-271ee6793414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:00.619221 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793bf66-d23b-4e79-a4cb-271ee6793414 admin admin] PATH_INFO: `/v3/projects/0a88369caf124bf9ad9b7edf3e145009/users/7cbaeeff37314ec7bfb8cd8e26243e30/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:00.621498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793bf66-d23b-4e79-a4cb-271ee6793414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a88369caf124bf9ad9b7edf3e145009, user_id=7cbaeeff37314ec7bfb8cd8e26243e30, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:00.625786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793bf66-d23b-4e79-a4cb-271ee6793414 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:00.638193 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1002/2020] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:00 2022] PUT /identity/v3/projects/0a88369caf124bf9ad9b7edf3e145009/users/7cbaeeff37314ec7bfb8cd8e26243e30/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:01.402116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad393d-ad9b-4b7c-98a0-04c249222f2d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:01.402639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad393d-ad9b-4b7c-98a0-04c249222f2d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:01.402750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad393d-ad9b-4b7c-98a0-04c249222f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:01.436903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a3ad393d-ad9b-4b7c-98a0-04c249222f2d None None] MFA Rules not processed for user `7cbaeeff37314ec7bfb8cd8e26243e30`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:01.438242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a3ad393d-ad9b-4b7c-98a0-04c249222f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:01.449607 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:01.522216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1019/2021] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:01.534153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793bf66-d23b-4e79-a4cb-271ee6793414 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:01.535932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9793bf66-d23b-4e79-a4cb-271ee6793414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:01.599036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793bf66-d23b-4e79-a4cb-271ee6793414 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:01.602727 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b12a5-9a3c-4d27-8883-c7832f23b596 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:01.603379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b12a5-9a3c-4d27-8883-c7832f23b596 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:01.603657 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b12a5-9a3c-4d27-8883-c7832f23b596 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:01.603951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b12a5-9a3c-4d27-8883-c7832f23b596 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:01.661882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b12a5-9a3c-4d27-8883-c7832f23b596 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:01.663348 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b12a5-9a3c-4d27-8883-c7832f23b596 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:01.672940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1003/2022] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:03.140579 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad393d-ad9b-4b7c-98a0-04c249222f2d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:03.142446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a3ad393d-ad9b-4b7c-98a0-04c249222f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:03.164558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad393d-ad9b-4b7c-98a0-04c249222f2d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:03.168278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9478d-d3a4-4175-a119-ad68fb575077 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:03.169131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9478d-d3a4-4175-a119-ad68fb575077 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:03.169413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9478d-d3a4-4175-a119-ad68fb575077 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:03.169690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9478d-d3a4-4175-a119-ad68fb575077 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:03.179300 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9478d-d3a4-4175-a119-ad68fb575077 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:03.180748 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9478d-d3a4-4175-a119-ad68fb575077 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:03.193475 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1020/2023] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:04.473133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b12a5-9a3c-4d27-8883-c7832f23b596 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:04.474652 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f40b12a5-9a3c-4d27-8883-c7832f23b596 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:04.550807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b12a5-9a3c-4d27-8883-c7832f23b596 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:04.555974 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:04.556691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:04.556975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:04.557253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] PATH_INFO: `/v3/users/d9431ce2c3964752a43047b5d51b57a0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:04.558398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9431ce2c3964752a43047b5d51b57a0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:04.560930 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:04.624596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9431ce2c3964752a43047b5d51b57a0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:04.628064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:04.643655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9431ce2c3964752a43047b5d51b57a0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:04.668051 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9431ce2c3964752a43047b5d51b57a0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:04.669522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9431ce2c3964752a43047b5d51b57a0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:04.679804 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9431ce2c3964752a43047b5d51b57a0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:04.690344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1004/2024] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:04 2022] DELETE /identity/v3/users/d9431ce2c3964752a43047b5d51b57a0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:04.696768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750e6eb-2f95-4cf3-b4ad-09811f09656b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:04.701119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750e6eb-2f95-4cf3-b4ad-09811f09656b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:04.701401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750e6eb-2f95-4cf3-b4ad-09811f09656b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:04.764487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4750e6eb-2f95-4cf3-b4ad-09811f09656b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:04.768633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4750e6eb-2f95-4cf3-b4ad-09811f09656b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:04.867102 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1021/2025] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:04.878057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:04.880395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:05.005383 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63286ad-f913-445d-90e5-036ab4e9664f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:05.009969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca312ab3-e216-4e6a-b2d0-0ea520053105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:05.010661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca312ab3-e216-4e6a-b2d0-0ea520053105 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:05.010899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca312ab3-e216-4e6a-b2d0-0ea520053105 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:05.011199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca312ab3-e216-4e6a-b2d0-0ea520053105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:05.022165 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca312ab3-e216-4e6a-b2d0-0ea520053105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:05.023755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca312ab3-e216-4e6a-b2d0-0ea520053105 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:05.034781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1005/2026] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:16:05.359657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750e6eb-2f95-4cf3-b4ad-09811f09656b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:05.361103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4750e6eb-2f95-4cf3-b4ad-09811f09656b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:05.373877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750e6eb-2f95-4cf3-b4ad-09811f09656b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:05.378594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336cfca-99b3-4dc2-a68d-e9b21232f570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:05.379270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336cfca-99b3-4dc2-a68d-e9b21232f570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:05.379557 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336cfca-99b3-4dc2-a68d-e9b21232f570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:05.379831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336cfca-99b3-4dc2-a68d-e9b21232f570 admin admin] PATH_INFO: `/v3/projects/94c9131322c74cdeb5ae7a514bec97db` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:05.381446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336cfca-99b3-4dc2-a68d-e9b21232f570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94c9131322c74cdeb5ae7a514bec97db)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:05.382895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336cfca-99b3-4dc2-a68d-e9b21232f570 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:05.430507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0336cfca-99b3-4dc2-a68d-e9b21232f570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94c9131322c74cdeb5ae7a514bec97db', 'request_id': 'req-0336cfca-99b3-4dc2-a68d-e9b21232f570'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:05.438693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0336cfca-99b3-4dc2-a68d-e9b21232f570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94c9131322c74cdeb5ae7a514bec97db', 'request_id': 'req-0336cfca-99b3-4dc2-a68d-e9b21232f570'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:05.444793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0336cfca-99b3-4dc2-a68d-e9b21232f570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94c9131322c74cdeb5ae7a514bec97db', 'request_id': 'req-0336cfca-99b3-4dc2-a68d-e9b21232f570'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:05.456500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0336cfca-99b3-4dc2-a68d-e9b21232f570 admin admin] The token cache is being invalidate because project 94c9131322c74cdeb5ae7a514bec97db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:05.456785 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0336cfca-99b3-4dc2-a68d-e9b21232f570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:05.457430 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0336cfca-99b3-4dc2-a68d-e9b21232f570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:05.458928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1022/2027] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:05 2022] DELETE /identity/v3/projects/94c9131322c74cdeb5ae7a514bec97db => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:05.463954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc6a92-c93a-40cc-97b2-517dd9c1bc80 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:05.464252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc6a92-c93a-40cc-97b2-517dd9c1bc80 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:05.464560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc6a92-c93a-40cc-97b2-517dd9c1bc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:05.484426 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4ffc6a92-c93a-40cc-97b2-517dd9c1bc80 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:05.485646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4ffc6a92-c93a-40cc-97b2-517dd9c1bc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:05.545279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1006/2028] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:05.552760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8d243-6f06-46c7-ad11-f273e6cd8bcb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:05.553051 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8d243-6f06-46c7-ad11-f273e6cd8bcb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:05.553332 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8d243-6f06-46c7-ad11-f273e6cd8bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:05.573530 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4be8d243-6f06-46c7-ad11-f273e6cd8bcb None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:05.575038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4be8d243-6f06-46c7-ad11-f273e6cd8bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:05.583215 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1023/2029] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:05.587481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc6a92-c93a-40cc-97b2-517dd9c1bc80 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:05.601433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc6a92-c93a-40cc-97b2-517dd9c1bc80 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:05.606855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e49e3-9d42-45d9-9f2b-b401febbac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:05.607509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e49e3-9d42-45d9-9f2b-b401febbac04 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:05.607783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e49e3-9d42-45d9-9f2b-b401febbac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:05.608049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e49e3-9d42-45d9-9f2b-b401febbac04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:05.608694 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e49e3-9d42-45d9-9f2b-b401febbac04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:16:05.609161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e49e3-9d42-45d9-9f2b-b401febbac04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:05.611028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e49e3-9d42-45d9-9f2b-b401febbac04 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:05.627083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1007/2030] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:16:05.628478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8d243-6f06-46c7-ad11-f273e6cd8bcb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:05.637952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8d243-6f06-46c7-ad11-f273e6cd8bcb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:05.642857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fa7b1-980f-4ed5-9a5b-9c00f22a1264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:05.643516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fa7b1-980f-4ed5-9a5b-9c00f22a1264 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:05.643919 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fa7b1-980f-4ed5-9a5b-9c00f22a1264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:05.644196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fa7b1-980f-4ed5-9a5b-9c00f22a1264 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:05.644842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fa7b1-980f-4ed5-9a5b-9c00f22a1264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:05.646744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fa7b1-980f-4ed5-9a5b-9c00f22a1264 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:05.652303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:05.680732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1024/2031] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:05 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:05.682322 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e49e3-9d42-45d9-9f2b-b401febbac04 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:05.693461 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e49e3-9d42-45d9-9f2b-b401febbac04 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:05.745474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:05.754471 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2477aa-db03-49e6-9e04-e15422e36776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:05.755279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2477aa-db03-49e6-9e04-e15422e36776 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:05.755763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2477aa-db03-49e6-9e04-e15422e36776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:05.756205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2477aa-db03-49e6-9e04-e15422e36776 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:05.757066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2477aa-db03-49e6-9e04-e15422e36776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:05.759021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2477aa-db03-49e6-9e04-e15422e36776 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:05.781458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1008/2032] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:05 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:16:05.785478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fa7b1-980f-4ed5-9a5b-9c00f22a1264 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:05.796123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fa7b1-980f-4ed5-9a5b-9c00f22a1264 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:05.801248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd89760-a350-4d56-87f2-8f300f52f869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:05.801880 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd89760-a350-4d56-87f2-8f300f52f869 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:05.802186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd89760-a350-4d56-87f2-8f300f52f869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:05.802459 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd89760-a350-4d56-87f2-8f300f52f869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:05.803138 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd89760-a350-4d56-87f2-8f300f52f869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:05.804735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd89760-a350-4d56-87f2-8f300f52f869 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:05.815602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1025/2033] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:05.819570 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2477aa-db03-49e6-9e04-e15422e36776 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:05.830532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2477aa-db03-49e6-9e04-e15422e36776 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:05.835264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b758a-f446-4346-b06e-27c739b9532f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:05.835814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b758a-f446-4346-b06e-27c739b9532f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:05.836093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b758a-f446-4346-b06e-27c739b9532f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:05.836369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b758a-f446-4346-b06e-27c739b9532f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:05.837049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b758a-f446-4346-b06e-27c739b9532f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:05.838770 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b758a-f446-4346-b06e-27c739b9532f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:05.853416 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1009/2034] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:05.854408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd89760-a350-4d56-87f2-8f300f52f869 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:05.864666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd89760-a350-4d56-87f2-8f300f52f869 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:05.869334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c75de-f0a5-4ae7-b41a-481f1a349335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:05.869972 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c75de-f0a5-4ae7-b41a-481f1a349335 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:05.870249 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c75de-f0a5-4ae7-b41a-481f1a349335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:05.870516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c75de-f0a5-4ae7-b41a-481f1a349335 admin admin] PATH_INFO: `/v3/projects/3e81d7a7e59c4daf825e7313e0a62763/users/44490a948a2b4d5db1e7ce04034cc275/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:05.885806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c75de-f0a5-4ae7-b41a-481f1a349335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e81d7a7e59c4daf825e7313e0a62763, user_id=44490a948a2b4d5db1e7ce04034cc275, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:05.889472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c75de-f0a5-4ae7-b41a-481f1a349335 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:05.903903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1026/2035] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:05 2022] PUT /identity/v3/projects/3e81d7a7e59c4daf825e7313e0a62763/users/44490a948a2b4d5db1e7ce04034cc275/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:05.914170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f98813-3e04-48a1-879e-1af9e6827460 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:05.914460 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f98813-3e04-48a1-879e-1af9e6827460 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:05.914815 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f98813-3e04-48a1-879e-1af9e6827460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:05.945231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d5f98813-3e04-48a1-879e-1af9e6827460 None None] MFA Rules not processed for user `44490a948a2b4d5db1e7ce04034cc275`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:05.946502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d5f98813-3e04-48a1-879e-1af9e6827460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:06.010323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c75de-f0a5-4ae7-b41a-481f1a349335 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:06.012871 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f68c75de-f0a5-4ae7-b41a-481f1a349335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:06.023870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1010/2036] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:06.040171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f98813-3e04-48a1-879e-1af9e6827460 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:06.041956 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d5f98813-3e04-48a1-879e-1af9e6827460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:06.128382 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c75de-f0a5-4ae7-b41a-481f1a349335 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:06.131688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f98813-3e04-48a1-879e-1af9e6827460 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:06.133859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:06.134512 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:06.134786 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:06.135080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] PATH_INFO: `/v3/users/6c148ee64f8b43d2a7bf879c30bcb8ef` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:06.136220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c148ee64f8b43d2a7bf879c30bcb8ef)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:06.136717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b91e46-59a6-4f9b-bb71-894d6a309893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:06.138124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b91e46-59a6-4f9b-bb71-894d6a309893 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:06.138499 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b91e46-59a6-4f9b-bb71-894d6a309893 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:06.139129 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b91e46-59a6-4f9b-bb71-894d6a309893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:06.139323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:06.149554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b91e46-59a6-4f9b-bb71-894d6a309893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:06.151250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b91e46-59a6-4f9b-bb71-894d6a309893 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:06.161424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1011/2037] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:16:06.236159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c148ee64f8b43d2a7bf879c30bcb8ef'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:06.253901 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c148ee64f8b43d2a7bf879c30bcb8ef'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:06.254700 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c148ee64f8b43d2a7bf879c30bcb8ef'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:06.284508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c148ee64f8b43d2a7bf879c30bcb8ef'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:06.311456 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c148ee64f8b43d2a7bf879c30bcb8ef'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:06.332013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1027/2038] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:06 2022] DELETE /identity/v3/users/6c148ee64f8b43d2a7bf879c30bcb8ef => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:16:06.338032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b88dd0-250f-445b-b0a3-93f1adbde99b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:06.339635 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b88dd0-250f-445b-b0a3-93f1adbde99b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:06.342080 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b88dd0-250f-445b-b0a3-93f1adbde99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:06.377194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-99b88dd0-250f-445b-b0a3-93f1adbde99b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:06.378617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-99b88dd0-250f-445b-b0a3-93f1adbde99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:06.458530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1012/2039] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:06.469339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:06.470887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:06.510397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:06.539312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c5b04-0fc3-4ecd-8b0a-18c6396dfdf7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:06.545832 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a6edf-6724-492b-b54d-324e87e294b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:06.546648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a6edf-6724-492b-b54d-324e87e294b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:06.547191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a6edf-6724-492b-b54d-324e87e294b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:06.547654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a6edf-6724-492b-b54d-324e87e294b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:06.557821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a6edf-6724-492b-b54d-324e87e294b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:06.559492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a6edf-6724-492b-b54d-324e87e294b3 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:06.569812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1028/2040] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:06.946556 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b88dd0-250f-445b-b0a3-93f1adbde99b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:06.954747 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-99b88dd0-250f-445b-b0a3-93f1adbde99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:06.982192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b88dd0-250f-445b-b0a3-93f1adbde99b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:06.988333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:06.989418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:06.989880 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:06.990002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] PATH_INFO: `/v3/projects/538f1079199a449b873d467088899c28` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:06.991042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=538f1079199a449b873d467088899c28)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:06.993075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:06.994908 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:07.050319 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '538f1079199a449b873d467088899c28', 'request_id': 'req-288877de-78ce-45cb-b1dd-1df07b352057'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:07.057633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '538f1079199a449b873d467088899c28', 'request_id': 'req-288877de-78ce-45cb-b1dd-1df07b352057'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:07.064999 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '538f1079199a449b873d467088899c28', 'request_id': 'req-288877de-78ce-45cb-b1dd-1df07b352057'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:07.078628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] The token cache is being invalidate because project 538f1079199a449b873d467088899c28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:07.079197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:07.080297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:07.081379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1013/2041] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:06 2022] DELETE /identity/v3/projects/538f1079199a449b873d467088899c28 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:07.085486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a6edf-6724-492b-b54d-324e87e294b3 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:07.096921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a6edf-6724-492b-b54d-324e87e294b3 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:07.162259 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88eb130-75b9-4a1c-919f-bc0e1f3f7a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:07.162903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88eb130-75b9-4a1c-919f-bc0e1f3f7a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:07.163229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88eb130-75b9-4a1c-919f-bc0e1f3f7a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:07.163501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88eb130-75b9-4a1c-919f-bc0e1f3f7a50 admin admin] PATH_INFO: `/v3/users/619f17dc752e4363a726a6b6121bffbb` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:07.164630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88eb130-75b9-4a1c-919f-bc0e1f3f7a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=619f17dc752e4363a726a6b6121bffbb)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:07.166561 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88eb130-75b9-4a1c-919f-bc0e1f3f7a50 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:07.218737 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b88eb130-75b9-4a1c-919f-bc0e1f3f7a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '619f17dc752e4363a726a6b6121bffbb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:07.227082 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b88eb130-75b9-4a1c-919f-bc0e1f3f7a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '619f17dc752e4363a726a6b6121bffbb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:07.227745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b88eb130-75b9-4a1c-919f-bc0e1f3f7a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '619f17dc752e4363a726a6b6121bffbb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:07.241788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b88eb130-75b9-4a1c-919f-bc0e1f3f7a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '619f17dc752e4363a726a6b6121bffbb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:07.249375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b88eb130-75b9-4a1c-919f-bc0e1f3f7a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '619f17dc752e4363a726a6b6121bffbb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:07.256945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1029/2042] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:07 2022] DELETE /identity/v3/users/619f17dc752e4363a726a6b6121bffbb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:07.895821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:07.897663 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:07.933665 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288877de-78ce-45cb-b1dd-1df07b352057 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:08.068463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfe88b-55fa-4572-95b5-8b84289a249b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:08.068463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfe88b-55fa-4572-95b5-8b84289a249b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.068463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfe88b-55fa-4572-95b5-8b84289a249b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.068463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfe88b-55fa-4572-95b5-8b84289a249b admin admin] PATH_INFO: `/v3/projects/6030e0bb28f44f2ca585f254fdb5bcac` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.069298 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cfe88b-55fa-4572-95b5-8b84289a249b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6030e0bb28f44f2ca585f254fdb5bcac)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:08.070323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cfe88b-55fa-4572-95b5-8b84289a249b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:08.130036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a4cfe88b-55fa-4572-95b5-8b84289a249b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6030e0bb28f44f2ca585f254fdb5bcac', 'request_id': 'req-a4cfe88b-55fa-4572-95b5-8b84289a249b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:08.136586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a4cfe88b-55fa-4572-95b5-8b84289a249b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6030e0bb28f44f2ca585f254fdb5bcac', 'request_id': 'req-a4cfe88b-55fa-4572-95b5-8b84289a249b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:08.144074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a4cfe88b-55fa-4572-95b5-8b84289a249b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6030e0bb28f44f2ca585f254fdb5bcac', 'request_id': 'req-a4cfe88b-55fa-4572-95b5-8b84289a249b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:08.159353 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a4cfe88b-55fa-4572-95b5-8b84289a249b admin admin] The token cache is being invalidate because project 6030e0bb28f44f2ca585f254fdb5bcac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:08.159745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a4cfe88b-55fa-4572-95b5-8b84289a249b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:08.160423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a4cfe88b-55fa-4572-95b5-8b84289a249b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:08.162176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1014/2043] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:07 2022] DELETE /identity/v3/projects/6030e0bb28f44f2ca585f254fdb5bcac => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:08.167138 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb40e9-eedb-4e7d-b35b-f04a42146033 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.167670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb40e9-eedb-4e7d-b35b-f04a42146033 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.167780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb40e9-eedb-4e7d-b35b-f04a42146033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.188438 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-50fb40e9-eedb-4e7d-b35b-f04a42146033 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:08.189663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-50fb40e9-eedb-4e7d-b35b-f04a42146033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:08.237457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:08.251717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1030/2044] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:08.259597 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9da45a-36ed-4316-a2fe-ff3a9304cd30 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.259884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9da45a-36ed-4316-a2fe-ff3a9304cd30 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.260161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9da45a-36ed-4316-a2fe-ff3a9304cd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.280961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3f9da45a-36ed-4316-a2fe-ff3a9304cd30 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:08.282207 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3f9da45a-36ed-4316-a2fe-ff3a9304cd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:08.290181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1015/2045] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:08.294491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb40e9-eedb-4e7d-b35b-f04a42146033 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:08.304379 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb40e9-eedb-4e7d-b35b-f04a42146033 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:08.309452 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ee339-d815-4186-9eb1-0767894f4e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:08.310058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ee339-d815-4186-9eb1-0767894f4e44 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.310349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ee339-d815-4186-9eb1-0767894f4e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.310577 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ee339-d815-4186-9eb1-0767894f4e44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.311313 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ee339-d815-4186-9eb1-0767894f4e44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:16:08.311618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ee339-d815-4186-9eb1-0767894f4e44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:08.313219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ee339-d815-4186-9eb1-0767894f4e44 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:08.325456 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1031/2046] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:16:08.329764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9da45a-36ed-4316-a2fe-ff3a9304cd30 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:08.338951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9da45a-36ed-4316-a2fe-ff3a9304cd30 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:08.344327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55be05-4bd3-4f6e-91cd-533006d2b172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:08.344928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55be05-4bd3-4f6e-91cd-533006d2b172 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.345204 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55be05-4bd3-4f6e-91cd-533006d2b172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.345477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55be05-4bd3-4f6e-91cd-533006d2b172 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.346188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f55be05-4bd3-4f6e-91cd-533006d2b172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:08.348121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f55be05-4bd3-4f6e-91cd-533006d2b172 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:08.359182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:08.377651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1016/2047] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:08 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:08.381778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ee339-d815-4186-9eb1-0767894f4e44 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:08.392490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ee339-d815-4186-9eb1-0767894f4e44 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:08.448598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57e523-df1f-4fe2-8932-199bbb5e352f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:08.449215 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57e523-df1f-4fe2-8932-199bbb5e352f admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.449497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57e523-df1f-4fe2-8932-199bbb5e352f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.450296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57e523-df1f-4fe2-8932-199bbb5e352f admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.450933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57e523-df1f-4fe2-8932-199bbb5e352f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:08.452885 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57e523-df1f-4fe2-8932-199bbb5e352f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:08.475106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1032/2048] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:08 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:08.478618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55be05-4bd3-4f6e-91cd-533006d2b172 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:08.489243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55be05-4bd3-4f6e-91cd-533006d2b172 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:08.493752 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85db22-1e82-4922-a61d-3e41fc1c6698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:08.494353 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85db22-1e82-4922-a61d-3e41fc1c6698 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.494630 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85db22-1e82-4922-a61d-3e41fc1c6698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.494905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85db22-1e82-4922-a61d-3e41fc1c6698 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.495576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85db22-1e82-4922-a61d-3e41fc1c6698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:08.497252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85db22-1e82-4922-a61d-3e41fc1c6698 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:08.507158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1017/2049] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:08.510980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57e523-df1f-4fe2-8932-199bbb5e352f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:08.521551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57e523-df1f-4fe2-8932-199bbb5e352f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:08.526242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784fb70-0d3d-4c8f-95af-d4b2dd7c1d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:08.526840 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784fb70-0d3d-4c8f-95af-d4b2dd7c1d84 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.527157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784fb70-0d3d-4c8f-95af-d4b2dd7c1d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.527435 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784fb70-0d3d-4c8f-95af-d4b2dd7c1d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.528329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784fb70-0d3d-4c8f-95af-d4b2dd7c1d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:08.529952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784fb70-0d3d-4c8f-95af-d4b2dd7c1d84 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:08.540541 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1033/2050] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:08.544843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85db22-1e82-4922-a61d-3e41fc1c6698 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:08.554610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85db22-1e82-4922-a61d-3e41fc1c6698 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:08.559629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f00c7-40de-4b30-8027-18d9969d1416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:08.560453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f00c7-40de-4b30-8027-18d9969d1416 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.560797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f00c7-40de-4b30-8027-18d9969d1416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.561097 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f00c7-40de-4b30-8027-18d9969d1416 admin admin] PATH_INFO: `/v3/projects/fcc963f990124bd3ae36e5e8d7d35a44/users/0116ccbb0b9f40f6895ae2ddf7283a85/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.576633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f00c7-40de-4b30-8027-18d9969d1416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcc963f990124bd3ae36e5e8d7d35a44, user_id=0116ccbb0b9f40f6895ae2ddf7283a85, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:08.579496 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f00c7-40de-4b30-8027-18d9969d1416 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:08.593251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1018/2051] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:08 2022] PUT /identity/v3/projects/fcc963f990124bd3ae36e5e8d7d35a44/users/0116ccbb0b9f40f6895ae2ddf7283a85/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:08.604589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691656a-af18-4109-9316-42a8ce7e3679 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.604902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691656a-af18-4109-9316-42a8ce7e3679 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.605183 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691656a-af18-4109-9316-42a8ce7e3679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.634253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1691656a-af18-4109-9316-42a8ce7e3679 None None] MFA Rules not processed for user `0116ccbb0b9f40f6895ae2ddf7283a85`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:08.635597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1691656a-af18-4109-9316-42a8ce7e3679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:08.703071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1034/2052] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:08.707500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f00c7-40de-4b30-8027-18d9969d1416 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:08.769304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f00c7-40de-4b30-8027-18d9969d1416 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:08.773923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fac959-26a8-401c-9d3e-ea75c919211d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:08.774526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fac959-26a8-401c-9d3e-ea75c919211d admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.774807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fac959-26a8-401c-9d3e-ea75c919211d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.775108 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fac959-26a8-401c-9d3e-ea75c919211d admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.775747 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fac959-26a8-401c-9d3e-ea75c919211d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:08.777659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fac959-26a8-401c-9d3e-ea75c919211d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:08.804410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1019/2053] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:08 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:08.808468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691656a-af18-4109-9316-42a8ce7e3679 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:08.819664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691656a-af18-4109-9316-42a8ce7e3679 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:08.864684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:08.881606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff7118-5600-4718-95e7-80bc3e320c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:08.882423 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7118-5600-4718-95e7-80bc3e320c88 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.882710 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7118-5600-4718-95e7-80bc3e320c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.882987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7118-5600-4718-95e7-80bc3e320c88 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.886279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff7118-5600-4718-95e7-80bc3e320c88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:08.888892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff7118-5600-4718-95e7-80bc3e320c88 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:08.923661 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1035/2054] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:08 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:08.926010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fac959-26a8-401c-9d3e-ea75c919211d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:08.936811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fac959-26a8-401c-9d3e-ea75c919211d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:08.941677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea53b7-2002-418c-b738-03776da80ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:08.942289 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea53b7-2002-418c-b738-03776da80ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.942565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea53b7-2002-418c-b738-03776da80ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.942837 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea53b7-2002-418c-b738-03776da80ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.943519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea53b7-2002-418c-b738-03776da80ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:08.945152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea53b7-2002-418c-b738-03776da80ca1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:08.957185 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1020/2055] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:08.961754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff7118-5600-4718-95e7-80bc3e320c88 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:08.973388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff7118-5600-4718-95e7-80bc3e320c88 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:08.978237 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e839e-6d45-440b-bb31-ef49f6b5b1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:08.978877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e839e-6d45-440b-bb31-ef49f6b5b1fa admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:08.979194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e839e-6d45-440b-bb31-ef49f6b5b1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:08.979472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e839e-6d45-440b-bb31-ef49f6b5b1fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:08.980119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e839e-6d45-440b-bb31-ef49f6b5b1fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:08.983317 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e839e-6d45-440b-bb31-ef49f6b5b1fa admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:08.994193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1036/2056] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:08.998965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea53b7-2002-418c-b738-03776da80ca1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:09.008925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea53b7-2002-418c-b738-03776da80ca1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:09.013634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21642b62-74cf-4d77-aca6-20fa2f8c2057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:09.014283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21642b62-74cf-4d77-aca6-20fa2f8c2057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:09.014564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21642b62-74cf-4d77-aca6-20fa2f8c2057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:09.014838 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21642b62-74cf-4d77-aca6-20fa2f8c2057 admin admin] PATH_INFO: `/v3/projects/78a5b4db64e241c5b9bddfb231af1b89/users/cb088f447463466abcf5c2a997ca18d7/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:09.029280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21642b62-74cf-4d77-aca6-20fa2f8c2057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a5b4db64e241c5b9bddfb231af1b89, user_id=cb088f447463466abcf5c2a997ca18d7, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:09.031909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21642b62-74cf-4d77-aca6-20fa2f8c2057 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:09.045737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1021/2057] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:08 2022] PUT /identity/v3/projects/78a5b4db64e241c5b9bddfb231af1b89/users/cb088f447463466abcf5c2a997ca18d7/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:16:09.058089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a162d-b8fa-49bb-a5fe-076f5ac2e959 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:09.058384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a162d-b8fa-49bb-a5fe-076f5ac2e959 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:09.059175 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a162d-b8fa-49bb-a5fe-076f5ac2e959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:09.090643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-894a162d-b8fa-49bb-a5fe-076f5ac2e959 None None] MFA Rules not processed for user `cb088f447463466abcf5c2a997ca18d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:09.092097 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-894a162d-b8fa-49bb-a5fe-076f5ac2e959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:09.166823 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1037/2058] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:09.172009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21642b62-74cf-4d77-aca6-20fa2f8c2057 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:09.236031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21642b62-74cf-4d77-aca6-20fa2f8c2057 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:09.241005 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc4b8c-0987-45e4-9d54-e77868b0dee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:09.241621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc4b8c-0987-45e4-9d54-e77868b0dee7 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:09.241899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc4b8c-0987-45e4-9d54-e77868b0dee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:09.242239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc4b8c-0987-45e4-9d54-e77868b0dee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:09.242822 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc4b8c-0987-45e4-9d54-e77868b0dee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:09.244872 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc4b8c-0987-45e4-9d54-e77868b0dee7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:09.257124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:09.276475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1022/2059] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:09 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:09.277745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a162d-b8fa-49bb-a5fe-076f5ac2e959 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:09.289121 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a162d-b8fa-49bb-a5fe-076f5ac2e959 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:09.346364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3accaeb-78ff-4116-9bd2-06a5441e4456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:09.347004 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3accaeb-78ff-4116-9bd2-06a5441e4456 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:09.347318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3accaeb-78ff-4116-9bd2-06a5441e4456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:09.347591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3accaeb-78ff-4116-9bd2-06a5441e4456 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:09.348214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3accaeb-78ff-4116-9bd2-06a5441e4456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:09.350145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3accaeb-78ff-4116-9bd2-06a5441e4456 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:09.370901 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1038/2060] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:09 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:16:09.375140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc4b8c-0987-45e4-9d54-e77868b0dee7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:09.385277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc4b8c-0987-45e4-9d54-e77868b0dee7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:09.390265 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630bc65-8054-4f93-963e-7f2f384da670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:09.390864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630bc65-8054-4f93-963e-7f2f384da670 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:09.391174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630bc65-8054-4f93-963e-7f2f384da670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:09.391449 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630bc65-8054-4f93-963e-7f2f384da670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:09.392105 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630bc65-8054-4f93-963e-7f2f384da670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:09.393676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630bc65-8054-4f93-963e-7f2f384da670 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:09.406187 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1023/2061] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:09.410151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3accaeb-78ff-4116-9bd2-06a5441e4456 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:09.420855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3accaeb-78ff-4116-9bd2-06a5441e4456 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:09.425514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11305858-3c16-4293-af72-ed2f484d9f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:09.426165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11305858-3c16-4293-af72-ed2f484d9f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:09.426449 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11305858-3c16-4293-af72-ed2f484d9f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:09.426723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11305858-3c16-4293-af72-ed2f484d9f7e admin admin] PATH_INFO: `/v3/projects/4c6a81316d704ea99f6022e77e87ec8d/users/ca2321decdd143cf973d08ae73cb295b/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:09.441442 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11305858-3c16-4293-af72-ed2f484d9f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6a81316d704ea99f6022e77e87ec8d, user_id=ca2321decdd143cf973d08ae73cb295b, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:09.444168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11305858-3c16-4293-af72-ed2f484d9f7e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:09.457320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1039/2062] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:09 2022] PUT /identity/v3/projects/4c6a81316d704ea99f6022e77e87ec8d/users/ca2321decdd143cf973d08ae73cb295b/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:09.461245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630bc65-8054-4f93-963e-7f2f384da670 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:09.471595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630bc65-8054-4f93-963e-7f2f384da670 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:09.531882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afcb4dc-2639-4feb-8760-db269074ea55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:09.532539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcb4dc-2639-4feb-8760-db269074ea55 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:09.532816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcb4dc-2639-4feb-8760-db269074ea55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:09.533086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcb4dc-2639-4feb-8760-db269074ea55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:09.533754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afcb4dc-2639-4feb-8760-db269074ea55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:09.535463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afcb4dc-2639-4feb-8760-db269074ea55 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:09.546004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1024/2063] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:09.550374 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11305858-3c16-4293-af72-ed2f484d9f7e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:09.560965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11305858-3c16-4293-af72-ed2f484d9f7e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:09.565628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812224c4-1b2b-4402-95cc-900a97291246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:09.566264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812224c4-1b2b-4402-95cc-900a97291246 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:09.566541 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812224c4-1b2b-4402-95cc-900a97291246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:09.566808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812224c4-1b2b-4402-95cc-900a97291246 admin admin] PATH_INFO: `/v3/projects/4c6a81316d704ea99f6022e77e87ec8d/users/ca2321decdd143cf973d08ae73cb295b/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:09.568730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812224c4-1b2b-4402-95cc-900a97291246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6a81316d704ea99f6022e77e87ec8d, user_id=ca2321decdd143cf973d08ae73cb295b, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:09.571369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812224c4-1b2b-4402-95cc-900a97291246 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:09.581896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1040/2064] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:09 2022] PUT /identity/v3/projects/4c6a81316d704ea99f6022e77e87ec8d/users/ca2321decdd143cf973d08ae73cb295b/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:09.592523 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8f679-bdc7-40d4-96c0-9593dfb16532 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:09.592811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8f679-bdc7-40d4-96c0-9593dfb16532 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:09.593230 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8f679-bdc7-40d4-96c0-9593dfb16532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:09.623340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-54f8f679-bdc7-40d4-96c0-9593dfb16532 None None] MFA Rules not processed for user `ca2321decdd143cf973d08ae73cb295b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:09.624569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-54f8f679-bdc7-40d4-96c0-9593dfb16532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:09.694433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1025/2065] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:09.706142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812224c4-1b2b-4402-95cc-900a97291246 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:09.707910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-812224c4-1b2b-4402-95cc-900a97291246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:09.719042 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:09.768805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812224c4-1b2b-4402-95cc-900a97291246 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:09.772489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175c0cb-eef9-4a9d-b171-3cd98b2bb83c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:09.773095 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175c0cb-eef9-4a9d-b171-3cd98b2bb83c service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:09.773372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175c0cb-eef9-4a9d-b171-3cd98b2bb83c service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:09.773643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175c0cb-eef9-4a9d-b171-3cd98b2bb83c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:09.831581 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175c0cb-eef9-4a9d-b171-3cd98b2bb83c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:09.832949 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175c0cb-eef9-4a9d-b171-3cd98b2bb83c service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:09.842536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1041/2066] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:12.645181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8f679-bdc7-40d4-96c0-9593dfb16532 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:12.648143 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-54f8f679-bdc7-40d4-96c0-9593dfb16532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:12.662983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8f679-bdc7-40d4-96c0-9593dfb16532 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:12.669302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bf602-2fcd-486e-a940-821d75a05d72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:12.670177 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bf602-2fcd-486e-a940-821d75a05d72 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:12.670433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bf602-2fcd-486e-a940-821d75a05d72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:12.671486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bf602-2fcd-486e-a940-821d75a05d72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:12.739510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61bf602-2fcd-486e-a940-821d75a05d72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:12.741046 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61bf602-2fcd-486e-a940-821d75a05d72 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:12.745808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:12.751546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1026/2067] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:13.922038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175c0cb-eef9-4a9d-b171-3cd98b2bb83c service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:13.930405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175c0cb-eef9-4a9d-b171-3cd98b2bb83c service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:13.934041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b81dc-e668-4b72-99ce-3d5eb33872bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:13.935211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b81dc-e668-4b72-99ce-3d5eb33872bd service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:13.935494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b81dc-e668-4b72-99ce-3d5eb33872bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:13.935766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b81dc-e668-4b72-99ce-3d5eb33872bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:13.944523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b81dc-e668-4b72-99ce-3d5eb33872bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:13.945943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b81dc-e668-4b72-99ce-3d5eb33872bd service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:13.955973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1042/2068] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:16.084630 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6c269-5b9d-4a02-a48c-f95f1d09a40a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:16.085214 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6c269-5b9d-4a02-a48c-f95f1d09a40a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:16.085691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6c269-5b9d-4a02-a48c-f95f1d09a40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:16.107977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0cd6c269-5b9d-4a02-a48c-f95f1d09a40a None None] MFA Rules not processed for user `d73fdae39484489facaf6831430cd4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:16.109438 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0cd6c269-5b9d-4a02-a48c-f95f1d09a40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:16.183013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1027/2069] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:19.021339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b81dc-e668-4b72-99ce-3d5eb33872bd service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:19.121011 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b81dc-e668-4b72-99ce-3d5eb33872bd service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:19.127592 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:19.128365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:19.128810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:19.129229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] PATH_INFO: `/v3/users/0116ccbb0b9f40f6895ae2ddf7283a85` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:19.130950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0116ccbb0b9f40f6895ae2ddf7283a85)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:19.135993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:19.211382 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0116ccbb0b9f40f6895ae2ddf7283a85'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:19.217717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:19.225525 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0116ccbb0b9f40f6895ae2ddf7283a85'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:19.226797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0116ccbb0b9f40f6895ae2ddf7283a85'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:19.247558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0116ccbb0b9f40f6895ae2ddf7283a85'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:19.263556 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0116ccbb0b9f40f6895ae2ddf7283a85'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:19.272177 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1043/2070] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:19 2022] DELETE /identity/v3/users/0116ccbb0b9f40f6895ae2ddf7283a85 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:19.277042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca4b25-f71f-442d-b15a-55b5f3c4706d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:19.277544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca4b25-f71f-442d-b15a-55b5f3c4706d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:19.277733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca4b25-f71f-442d-b15a-55b5f3c4706d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:19.303556 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d9ca4b25-f71f-442d-b15a-55b5f3c4706d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:19.305383 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d9ca4b25-f71f-442d-b15a-55b5f3c4706d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:19.397613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1028/2071] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:19.442211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:19.446023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:19.578731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f05a35-7d0a-4e7e-ad47-2bbd614ecb83 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:19.584644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563a99a-363a-43a7-b541-8c59a894500f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:19.586722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563a99a-363a-43a7-b541-8c59a894500f service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:19.587242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563a99a-363a-43a7-b541-8c59a894500f service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:19.588514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563a99a-363a-43a7-b541-8c59a894500f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:19.613768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563a99a-363a-43a7-b541-8c59a894500f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:19.618722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563a99a-363a-43a7-b541-8c59a894500f service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:19.629745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1044/2072] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:20.750402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca4b25-f71f-442d-b15a-55b5f3c4706d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:20.751989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d9ca4b25-f71f-442d-b15a-55b5f3c4706d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:20.765057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca4b25-f71f-442d-b15a-55b5f3c4706d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:20.770120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe2571-89bd-462a-be52-8cc496cecd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:20.770766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe2571-89bd-462a-be52-8cc496cecd26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:20.771067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe2571-89bd-462a-be52-8cc496cecd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:20.771378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe2571-89bd-462a-be52-8cc496cecd26 admin admin] PATH_INFO: `/v3/projects/fcc963f990124bd3ae36e5e8d7d35a44` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:20.780743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe2571-89bd-462a-be52-8cc496cecd26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcc963f990124bd3ae36e5e8d7d35a44)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:20.780743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe2571-89bd-462a-be52-8cc496cecd26 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:20.835638 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d1fe2571-89bd-462a-be52-8cc496cecd26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcc963f990124bd3ae36e5e8d7d35a44', 'request_id': 'req-d1fe2571-89bd-462a-be52-8cc496cecd26'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:20.842696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d1fe2571-89bd-462a-be52-8cc496cecd26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcc963f990124bd3ae36e5e8d7d35a44', 'request_id': 'req-d1fe2571-89bd-462a-be52-8cc496cecd26'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:20.849130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d1fe2571-89bd-462a-be52-8cc496cecd26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcc963f990124bd3ae36e5e8d7d35a44', 'request_id': 'req-d1fe2571-89bd-462a-be52-8cc496cecd26'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:20.869972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d1fe2571-89bd-462a-be52-8cc496cecd26 admin admin] The token cache is being invalidate because project fcc963f990124bd3ae36e5e8d7d35a44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:20.870276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d1fe2571-89bd-462a-be52-8cc496cecd26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:20.870919 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d1fe2571-89bd-462a-be52-8cc496cecd26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:20.872921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1029/2073] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:20 2022] DELETE /identity/v3/projects/fcc963f990124bd3ae36e5e8d7d35a44 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:20.877445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563a99a-363a-43a7-b541-8c59a894500f service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:20.887034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563a99a-363a-43a7-b541-8c59a894500f service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:20.947008 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513329a4-f0cc-4b56-bd1e-18234c85f717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:20.947677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513329a4-f0cc-4b56-bd1e-18234c85f717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:20.947956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513329a4-f0cc-4b56-bd1e-18234c85f717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:20.948230 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513329a4-f0cc-4b56-bd1e-18234c85f717 admin admin] PATH_INFO: `/v3/users/cb088f447463466abcf5c2a997ca18d7` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:20.949411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513329a4-f0cc-4b56-bd1e-18234c85f717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb088f447463466abcf5c2a997ca18d7)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:20.952019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513329a4-f0cc-4b56-bd1e-18234c85f717 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:20.999488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-513329a4-f0cc-4b56-bd1e-18234c85f717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb088f447463466abcf5c2a997ca18d7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.006796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-513329a4-f0cc-4b56-bd1e-18234c85f717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb088f447463466abcf5c2a997ca18d7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.007454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-513329a4-f0cc-4b56-bd1e-18234c85f717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb088f447463466abcf5c2a997ca18d7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.020158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-513329a4-f0cc-4b56-bd1e-18234c85f717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb088f447463466abcf5c2a997ca18d7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.022166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:21.030346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-513329a4-f0cc-4b56-bd1e-18234c85f717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb088f447463466abcf5c2a997ca18d7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.036764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1045/2074] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:20 2022] DELETE /identity/v3/users/cb088f447463466abcf5c2a997ca18d7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:21.279565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc32095-c62b-46c7-bbd4-2d335ac6b586 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:21.280894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc32095-c62b-46c7-bbd4-2d335ac6b586 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:21.281479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc32095-c62b-46c7-bbd4-2d335ac6b586 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:21.284154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1030/2075] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:16:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:16:21.292807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b9063-846b-456e-91d8-8907ca4f82f7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:21.293187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b9063-846b-456e-91d8-8907ca4f82f7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:21.293538 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b9063-846b-456e-91d8-8907ca4f82f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:21.317241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9f8b9063-846b-456e-91d8-8907ca4f82f7 None None] MFA Rules not processed for user `dbc7cef7b714463697dc1ee507e60be1`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:21.318534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9f8b9063-846b-456e-91d8-8907ca4f82f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:21.388718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1046/2076] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:16:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:21.413085 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc32095-c62b-46c7-bbd4-2d335ac6b586 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:21.414556 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dbc32095-c62b-46c7-bbd4-2d335ac6b586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:21.419614 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b9063-846b-456e-91d8-8907ca4f82f7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:21.421375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9f8b9063-846b-456e-91d8-8907ca4f82f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:21.435851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b9063-846b-456e-91d8-8907ca4f82f7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:21.442049 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8df56-74d5-4bd1-83b2-cb410e549e24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:21.442667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8df56-74d5-4bd1-83b2-cb410e549e24 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:21.442943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8df56-74d5-4bd1-83b2-cb410e549e24 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:21.443433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8df56-74d5-4bd1-83b2-cb410e549e24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:21.452625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:21.454608 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8df56-74d5-4bd1-83b2-cb410e549e24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:21.456536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8df56-74d5-4bd1-83b2-cb410e549e24 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:21.468569 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1047/2077] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:21.470957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8df56-74d5-4bd1-83b2-cb410e549e24 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:21.472542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-13d8df56-74d5-4bd1-83b2-cb410e549e24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:21.521048 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc32095-c62b-46c7-bbd4-2d335ac6b586 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:21.525669 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:21.526306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:21.526582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:21.526859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] PATH_INFO: `/v3/users/efeb17c5a72b4e9294a63cf185737f3f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:21.528084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] RBAC: Authorizing `identity:delete_user(user_id=efeb17c5a72b4e9294a63cf185737f3f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:21.530154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:21.546891 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8df56-74d5-4bd1-83b2-cb410e549e24 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:21.551566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:21.552199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:21.552477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:21.552745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] PATH_INFO: `/v3/projects/78a5b4db64e241c5b9bddfb231af1b89` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:21.553860 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] RBAC: Authorizing `identity:delete_project(project_id=78a5b4db64e241c5b9bddfb231af1b89)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:21.556075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:21.580352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efeb17c5a72b4e9294a63cf185737f3f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.590151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efeb17c5a72b4e9294a63cf185737f3f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.612112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efeb17c5a72b4e9294a63cf185737f3f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.612839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efeb17c5a72b4e9294a63cf185737f3f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.613780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78a5b4db64e241c5b9bddfb231af1b89', 'request_id': 'req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.620422 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efeb17c5a72b4e9294a63cf185737f3f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.623024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78a5b4db64e241c5b9bddfb231af1b89', 'request_id': 'req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.627223 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1031/2078] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:21 2022] DELETE /identity/v3/users/efeb17c5a72b4e9294a63cf185737f3f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:21.628769 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78a5b4db64e241c5b9bddfb231af1b89', 'request_id': 'req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.641753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] The token cache is being invalidate because project 78a5b4db64e241c5b9bddfb231af1b89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:21.642044 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.642719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.644460 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1048/2079] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:21 2022] DELETE /identity/v3/projects/78a5b4db64e241c5b9bddfb231af1b89 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:21.648241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:21.649731 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:21.726122 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086fc5c-d5b1-46d1-80ae-2c7ef62d3cda admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:21.731828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6b4b9-08e9-4a8c-b256-c8e0529b1128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:21.732472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6b4b9-08e9-4a8c-b256-c8e0529b1128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:21.732936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6b4b9-08e9-4a8c-b256-c8e0529b1128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:21.733240 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6b4b9-08e9-4a8c-b256-c8e0529b1128 admin admin] PATH_INFO: `/v3/users/ca2321decdd143cf973d08ae73cb295b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:21.734596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6b4b9-08e9-4a8c-b256-c8e0529b1128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca2321decdd143cf973d08ae73cb295b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:21.736578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6b4b9-08e9-4a8c-b256-c8e0529b1128 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:21.808946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-37b6b4b9-08e9-4a8c-b256-c8e0529b1128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca2321decdd143cf973d08ae73cb295b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.812556 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:21.822628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-37b6b4b9-08e9-4a8c-b256-c8e0529b1128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca2321decdd143cf973d08ae73cb295b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.823730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:21.826881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:21.843009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-37b6b4b9-08e9-4a8c-b256-c8e0529b1128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca2321decdd143cf973d08ae73cb295b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.843751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-37b6b4b9-08e9-4a8c-b256-c8e0529b1128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca2321decdd143cf973d08ae73cb295b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.850202 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:21.854590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-37b6b4b9-08e9-4a8c-b256-c8e0529b1128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca2321decdd143cf973d08ae73cb295b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:21.866190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1032/2080] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:21 2022] DELETE /identity/v3/users/ca2321decdd143cf973d08ae73cb295b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:21.922096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dac9f-accb-4c7f-b0ef-68eec6e3aefe admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:21.982159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:21.982817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:21.983133 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:21.983405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] PATH_INFO: `/v3/projects/80ecff6bf26f4cbeadb78b7272febb35` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:21.984521 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ecff6bf26f4cbeadb78b7272febb35)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:21.986457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.041386 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ecff6bf26f4cbeadb78b7272febb35', 'request_id': 'req-68956f12-791d-4e85-ab38-49dd7648c1b2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:22.049499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ecff6bf26f4cbeadb78b7272febb35', 'request_id': 'req-68956f12-791d-4e85-ab38-49dd7648c1b2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:22.056937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ecff6bf26f4cbeadb78b7272febb35', 'request_id': 'req-68956f12-791d-4e85-ab38-49dd7648c1b2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:22.069527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] The token cache is being invalidate because project 80ecff6bf26f4cbeadb78b7272febb35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:22.069816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:22.070578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:22.072153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1049/2081] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:21 2022] DELETE /identity/v3/projects/80ecff6bf26f4cbeadb78b7272febb35 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:22.077942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e00a8-1ba0-42b4-b585-08258c8c73ef None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.078356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e00a8-1ba0-42b4-b585-08258c8c73ef None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.078773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e00a8-1ba0-42b4-b585-08258c8c73ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.098855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e68e00a8-1ba0-42b4-b585-08258c8c73ef None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:22.100209 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e68e00a8-1ba0-42b4-b585-08258c8c73ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:22.147909 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.149340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:22.181197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1033/2082] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:22.183179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c41af4-7aae-4867-a15d-0cfe810d668f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.183587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c41af4-7aae-4867-a15d-0cfe810d668f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.184635 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c41af4-7aae-4867-a15d-0cfe810d668f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.201503 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:22.204905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e1c41af4-7aae-4867-a15d-0cfe810d668f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:22.206115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e1c41af4-7aae-4867-a15d-0cfe810d668f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:22.214224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1034/2083] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:22.218906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c41af4-7aae-4867-a15d-0cfe810d668f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.224184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68956f12-791d-4e85-ab38-49dd7648c1b2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.228970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c41af4-7aae-4867-a15d-0cfe810d668f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.230072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13465b80-f526-4e29-94e2-44e8516c66c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.232135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13465b80-f526-4e29-94e2-44e8516c66c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.232422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13465b80-f526-4e29-94e2-44e8516c66c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.232695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13465b80-f526-4e29-94e2-44e8516c66c5 admin admin] PATH_INFO: `/v3/projects/4c6a81316d704ea99f6022e77e87ec8d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.233391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcf238-ea0c-4c16-9dde-e852d97bf962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.233814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13465b80-f526-4e29-94e2-44e8516c66c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c6a81316d704ea99f6022e77e87ec8d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.234004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcf238-ea0c-4c16-9dde-e852d97bf962 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.234277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcf238-ea0c-4c16-9dde-e852d97bf962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.234548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcf238-ea0c-4c16-9dde-e852d97bf962 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.235217 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcf238-ea0c-4c16-9dde-e852d97bf962 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:16:22.235520 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcf238-ea0c-4c16-9dde-e852d97bf962 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.235800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13465b80-f526-4e29-94e2-44e8516c66c5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.237090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcf238-ea0c-4c16-9dde-e852d97bf962 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.248962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1035/2084] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:16:22.253203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcf238-ea0c-4c16-9dde-e852d97bf962 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.263250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcf238-ea0c-4c16-9dde-e852d97bf962 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.268619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff10d0-ffdf-4735-8571-cb8c517612ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.269230 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff10d0-ffdf-4735-8571-cb8c517612ac admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.269507 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff10d0-ffdf-4735-8571-cb8c517612ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.269778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff10d0-ffdf-4735-8571-cb8c517612ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.270402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff10d0-ffdf-4735-8571-cb8c517612ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.272410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff10d0-ffdf-4735-8571-cb8c517612ac admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.285709 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-13465b80-f526-4e29-94e2-44e8516c66c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c6a81316d704ea99f6022e77e87ec8d', 'request_id': 'req-13465b80-f526-4e29-94e2-44e8516c66c5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:22.295121 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-13465b80-f526-4e29-94e2-44e8516c66c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c6a81316d704ea99f6022e77e87ec8d', 'request_id': 'req-13465b80-f526-4e29-94e2-44e8516c66c5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:22.301184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-13465b80-f526-4e29-94e2-44e8516c66c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c6a81316d704ea99f6022e77e87ec8d', 'request_id': 'req-13465b80-f526-4e29-94e2-44e8516c66c5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:22.304283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1036/2085] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:22 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:16:22.308320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff10d0-ffdf-4735-8571-cb8c517612ac admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.314041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-13465b80-f526-4e29-94e2-44e8516c66c5 admin admin] The token cache is being invalidate because project 4c6a81316d704ea99f6022e77e87ec8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:22.314466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-13465b80-f526-4e29-94e2-44e8516c66c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:22.315170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-13465b80-f526-4e29-94e2-44e8516c66c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:22.320043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1050/2086] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:22 2022] DELETE /identity/v3/projects/4c6a81316d704ea99f6022e77e87ec8d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:16:22.321674 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35627fb-84cb-4613-bf4d-81dcb0cc0f17 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.321961 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35627fb-84cb-4613-bf4d-81dcb0cc0f17 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.322431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35627fb-84cb-4613-bf4d-81dcb0cc0f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.362255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d35627fb-84cb-4613-bf4d-81dcb0cc0f17 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:22.363602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d35627fb-84cb-4613-bf4d-81dcb0cc0f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:22.396921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff10d0-ffdf-4735-8571-cb8c517612ac admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.403598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecb4e7-cac0-4bcc-8168-98cafa5510a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.404329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecb4e7-cac0-4bcc-8168-98cafa5510a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.404666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecb4e7-cac0-4bcc-8168-98cafa5510a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.405014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecb4e7-cac0-4bcc-8168-98cafa5510a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.405658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecb4e7-cac0-4bcc-8168-98cafa5510a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.407656 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecb4e7-cac0-4bcc-8168-98cafa5510a7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.436317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1037/2087] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:22 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:16:22.442083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecb4e7-cac0-4bcc-8168-98cafa5510a7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.442589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1051/2088] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:16:22.444283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-daecb4e7-cac0-4bcc-8168-98cafa5510a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:22.450632 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbb22d-3a22-4801-a572-84ce90908263 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.451335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbb22d-3a22-4801-a572-84ce90908263 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.451890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbb22d-3a22-4801-a572-84ce90908263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.461447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecb4e7-cac0-4bcc-8168-98cafa5510a7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.466602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c00046-ee15-4a2d-b8e5-1e06f103cb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.467259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c00046-ee15-4a2d-b8e5-1e06f103cb51 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.467545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c00046-ee15-4a2d-b8e5-1e06f103cb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.467821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c00046-ee15-4a2d-b8e5-1e06f103cb51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.468474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c00046-ee15-4a2d-b8e5-1e06f103cb51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.470145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c00046-ee15-4a2d-b8e5-1e06f103cb51 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.478164 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d9fbb22d-3a22-4801-a572-84ce90908263 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:22.481425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1038/2089] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:22.483191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d9fbb22d-3a22-4801-a572-84ce90908263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:22.485618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c00046-ee15-4a2d-b8e5-1e06f103cb51 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.492618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:22.493399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1052/2090] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:22.497928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbb22d-3a22-4801-a572-84ce90908263 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.498526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c00046-ee15-4a2d-b8e5-1e06f103cb51 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.505824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2748b-3ed7-41ce-a9d3-893e86c91798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.506521 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2748b-3ed7-41ce-a9d3-893e86c91798 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.506878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2748b-3ed7-41ce-a9d3-893e86c91798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.508116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2748b-3ed7-41ce-a9d3-893e86c91798 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.508777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbb22d-3a22-4801-a572-84ce90908263 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.508968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2748b-3ed7-41ce-a9d3-893e86c91798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.510854 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2748b-3ed7-41ce-a9d3-893e86c91798 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.514997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf586c5-7e21-4f7c-8050-2e1e2229ff2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.515680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf586c5-7e21-4f7c-8050-2e1e2229ff2f admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.516025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf586c5-7e21-4f7c-8050-2e1e2229ff2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.516760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf586c5-7e21-4f7c-8050-2e1e2229ff2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.518127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf586c5-7e21-4f7c-8050-2e1e2229ff2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:16:22.518363 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf586c5-7e21-4f7c-8050-2e1e2229ff2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.519997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf586c5-7e21-4f7c-8050-2e1e2229ff2f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.522164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1039/2091] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:16:22.526729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2748b-3ed7-41ce-a9d3-893e86c91798 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.538080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1053/2092] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:16:22.538231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2748b-3ed7-41ce-a9d3-893e86c91798 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.539007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf586c5-7e21-4f7c-8050-2e1e2229ff2f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.542788 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf45eb-dd84-4ded-a9cf-df446ef93e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.544045 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf45eb-dd84-4ded-a9cf-df446ef93e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.544495 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf45eb-dd84-4ded-a9cf-df446ef93e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.544811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf45eb-dd84-4ded-a9cf-df446ef93e5e admin admin] PATH_INFO: `/v3/projects/92f354242a7b4ae4b774681eb633c2b2/users/ca9eb6b9dc6e409a954c7993ebe5a5c3/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.549425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf586c5-7e21-4f7c-8050-2e1e2229ff2f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.554713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae3621-342e-4def-8998-44af195527c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.555320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3621-342e-4def-8998-44af195527c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.555600 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3621-342e-4def-8998-44af195527c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.555873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3621-342e-4def-8998-44af195527c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.556495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae3621-342e-4def-8998-44af195527c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.558428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae3621-342e-4def-8998-44af195527c9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.560340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf45eb-dd84-4ded-a9cf-df446ef93e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f354242a7b4ae4b774681eb633c2b2, user_id=ca9eb6b9dc6e409a954c7993ebe5a5c3, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.562977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf45eb-dd84-4ded-a9cf-df446ef93e5e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.578130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1040/2093] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:22 2022] PUT /identity/v3/projects/92f354242a7b4ae4b774681eb633c2b2/users/ca9eb6b9dc6e409a954c7993ebe5a5c3/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:16:22.583737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3517c37-ff10-4a59-89f5-d5699ecc3324 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.584006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3517c37-ff10-4a59-89f5-d5699ecc3324 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.584277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3517c37-ff10-4a59-89f5-d5699ecc3324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.589284 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1054/2094] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:22 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:16:22.597220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae3621-342e-4def-8998-44af195527c9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.603937 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e3517c37-ff10-4a59-89f5-d5699ecc3324 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:22.605308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e3517c37-ff10-4a59-89f5-d5699ecc3324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:22.638142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:22.674444 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae3621-342e-4def-8998-44af195527c9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.679971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf2fe5-3cb4-4e5d-b5fc-4a3c2c01b2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.680602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf2fe5-3cb4-4e5d-b5fc-4a3c2c01b2c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.680873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf2fe5-3cb4-4e5d-b5fc-4a3c2c01b2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.681147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf2fe5-3cb4-4e5d-b5fc-4a3c2c01b2c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.681786 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf2fe5-3cb4-4e5d-b5fc-4a3c2c01b2c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.682616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1041/2095] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:16:22.683799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf2fe5-3cb4-4e5d-b5fc-4a3c2c01b2c4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.692956 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3517c37-ff10-4a59-89f5-d5699ecc3324 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.694525 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e3517c37-ff10-4a59-89f5-d5699ecc3324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:22.711101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1055/2096] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:22 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:22.714346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf2fe5-3cb4-4e5d-b5fc-4a3c2c01b2c4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.725873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf2fe5-3cb4-4e5d-b5fc-4a3c2c01b2c4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.730734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a924be-9487-4d7b-b626-43b3f6e2e43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.731966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a924be-9487-4d7b-b626-43b3f6e2e43a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.732180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a924be-9487-4d7b-b626-43b3f6e2e43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.732773 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a924be-9487-4d7b-b626-43b3f6e2e43a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.733340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a924be-9487-4d7b-b626-43b3f6e2e43a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.735239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a924be-9487-4d7b-b626-43b3f6e2e43a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.748613 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1056/2097] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:22.752628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a924be-9487-4d7b-b626-43b3f6e2e43a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.764249 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a924be-9487-4d7b-b626-43b3f6e2e43a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.769357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfd83f-f1d7-4af9-b380-cfbbc36d7620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.769892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfd83f-f1d7-4af9-b380-cfbbc36d7620 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.770168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfd83f-f1d7-4af9-b380-cfbbc36d7620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.770439 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfd83f-f1d7-4af9-b380-cfbbc36d7620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.771020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3517c37-ff10-4a59-89f5-d5699ecc3324 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.771172 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfd83f-f1d7-4af9-b380-cfbbc36d7620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.772688 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfd83f-f1d7-4af9-b380-cfbbc36d7620 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.775777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613d792-7a16-43ff-9529-c587e9e1e3a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.776524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613d792-7a16-43ff-9529-c587e9e1e3a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.776885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613d792-7a16-43ff-9529-c587e9e1e3a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.777162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613d792-7a16-43ff-9529-c587e9e1e3a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.784127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1057/2098] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:22.788200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfd83f-f1d7-4af9-b380-cfbbc36d7620 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:22.791845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613d792-7a16-43ff-9529-c587e9e1e3a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.793375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613d792-7a16-43ff-9529-c587e9e1e3a0 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.799592 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfd83f-f1d7-4af9-b380-cfbbc36d7620 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:22.805374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1042/2099] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:16:22.814948 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8349eb-ab13-4097-ab70-3212c2f78e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:22.815722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8349eb-ab13-4097-ab70-3212c2f78e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.816013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8349eb-ab13-4097-ab70-3212c2f78e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.817768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8349eb-ab13-4097-ab70-3212c2f78e69 admin admin] PATH_INFO: `/v3/projects/8cbb8921820b4b5abc78e3d1b9af1a07/users/e43561ddb9e9400188652784415777c9/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.835332 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8349eb-ab13-4097-ab70-3212c2f78e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbb8921820b4b5abc78e3d1b9af1a07, user_id=e43561ddb9e9400188652784415777c9, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:22.838052 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8349eb-ab13-4097-ab70-3212c2f78e69 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:22.862087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1058/2100] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:22 2022] PUT /identity/v3/projects/8cbb8921820b4b5abc78e3d1b9af1a07/users/e43561ddb9e9400188652784415777c9/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:16:22.874023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd774a5-87b3-466c-84f7-3446ddbf10e9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:22.874592 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd774a5-87b3-466c-84f7-3446ddbf10e9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:22.874975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd774a5-87b3-466c-84f7-3446ddbf10e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:22.919910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cdd774a5-87b3-466c-84f7-3446ddbf10e9 None None] MFA Rules not processed for user `e43561ddb9e9400188652784415777c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:22.921289 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cdd774a5-87b3-466c-84f7-3446ddbf10e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:23.034185 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1043/2101] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:23.041593 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8349eb-ab13-4097-ab70-3212c2f78e69 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:23.045055 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fa8349eb-ab13-4097-ab70-3212c2f78e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:23.135520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8349eb-ab13-4097-ab70-3212c2f78e69 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:23.139715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fa916-a9d5-4d3b-b1e1-3ee9f531750a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:23.140339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fa916-a9d5-4d3b-b1e1-3ee9f531750a service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:23.141163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fa916-a9d5-4d3b-b1e1-3ee9f531750a service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:23.141163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fa916-a9d5-4d3b-b1e1-3ee9f531750a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:23.155778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fa916-a9d5-4d3b-b1e1-3ee9f531750a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:23.156818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fa916-a9d5-4d3b-b1e1-3ee9f531750a service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:23.167352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1059/2102] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:16:25.987097 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd774a5-87b3-466c-84f7-3446ddbf10e9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:25.992308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:26.108334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd774a5-87b3-466c-84f7-3446ddbf10e9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:26.118073 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbcf3c-6d23-437d-a928-afeda37b47ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:26.119253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbcf3c-6d23-437d-a928-afeda37b47ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:26.119531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbcf3c-6d23-437d-a928-afeda37b47ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:26.120089 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbcf3c-6d23-437d-a928-afeda37b47ba admin admin] PATH_INFO: `/v3/users/e43561ddb9e9400188652784415777c9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:26.121313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbcf3c-6d23-437d-a928-afeda37b47ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e43561ddb9e9400188652784415777c9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:26.124999 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbcf3c-6d23-437d-a928-afeda37b47ba admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:26.207874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b7dbcf3c-6d23-437d-a928-afeda37b47ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e43561ddb9e9400188652784415777c9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:26.221009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72218395-f912-4e87-9ec5-3b3704d8d7b7 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:26.221588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72218395-f912-4e87-9ec5-3b3704d8d7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:26.221588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72218395-f912-4e87-9ec5-3b3704d8d7b7 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:26.223266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1060/2103] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:16:26.224069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b7dbcf3c-6d23-437d-a928-afeda37b47ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e43561ddb9e9400188652784415777c9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:26.241258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7a4e1-bc4d-4deb-853a-663ad6331da3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:26.241680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7a4e1-bc4d-4deb-853a-663ad6331da3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:26.242027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7a4e1-bc4d-4deb-853a-663ad6331da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:26.261495 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b7dbcf3c-6d23-437d-a928-afeda37b47ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e43561ddb9e9400188652784415777c9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:26.262764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b7dbcf3c-6d23-437d-a928-afeda37b47ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e43561ddb9e9400188652784415777c9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:26.268137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-75a7a4e1-bc4d-4deb-853a-663ad6331da3 None None] MFA Rules not processed for user `dbc7cef7b714463697dc1ee507e60be1`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:26.269767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-75a7a4e1-bc4d-4deb-853a-663ad6331da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:26.271976 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b7dbcf3c-6d23-437d-a928-afeda37b47ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e43561ddb9e9400188652784415777c9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:26.291087 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1044/2104] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:25 2022] DELETE /identity/v3/users/e43561ddb9e9400188652784415777c9 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:26.292815 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e5193-aca6-4c15-8038-ecef9481b88c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:26.294640 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e5193-aca6-4c15-8038-ecef9481b88c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:26.295023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e5193-aca6-4c15-8038-ecef9481b88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:26.326950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9e5e5193-aca6-4c15-8038-ecef9481b88c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:26.339637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9e5e5193-aca6-4c15-8038-ecef9481b88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:26.370830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1061/2105] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:16:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:16:26.386876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7a4e1-bc4d-4deb-853a-663ad6331da3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:26.396756 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-75a7a4e1-bc4d-4deb-853a-663ad6331da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:26.420915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7a4e1-bc4d-4deb-853a-663ad6331da3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:26.474959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:26.483086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1045/2106] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:26.494701 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e5193-aca6-4c15-8038-ecef9481b88c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:26.498134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9e5e5193-aca6-4c15-8038-ecef9481b88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:26.555165 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:26.626530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e5193-aca6-4c15-8038-ecef9481b88c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:26.629506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1c8d1-68c5-47e3-8b0b-1906390edbd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:26.631084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da365e-1f82-46f2-a8bc-7aa1ea6d2258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:26.631226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1c8d1-68c5-47e3-8b0b-1906390edbd8 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:26.631587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1c8d1-68c5-47e3-8b0b-1906390edbd8 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:26.632340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1c8d1-68c5-47e3-8b0b-1906390edbd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:26.632650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da365e-1f82-46f2-a8bc-7aa1ea6d2258 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:26.632850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da365e-1f82-46f2-a8bc-7aa1ea6d2258 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:26.633131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da365e-1f82-46f2-a8bc-7aa1ea6d2258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:26.644739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da365e-1f82-46f2-a8bc-7aa1ea6d2258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:26.646362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1c8d1-68c5-47e3-8b0b-1906390edbd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:26.646491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da365e-1f82-46f2-a8bc-7aa1ea6d2258 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:26.648639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1c8d1-68c5-47e3-8b0b-1906390edbd8 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:26.660352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1046/2107] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:16:26.662730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1062/2108] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:27.100882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da365e-1f82-46f2-a8bc-7aa1ea6d2258 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:27.102392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f9da365e-1f82-46f2-a8bc-7aa1ea6d2258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:27.124063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da365e-1f82-46f2-a8bc-7aa1ea6d2258 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:27.132964 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:27.134110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:27.134809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:27.135361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] PATH_INFO: `/v3/users/44490a948a2b4d5db1e7ce04034cc275` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:27.136719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44490a948a2b4d5db1e7ce04034cc275)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:27.140197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:27.228411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44490a948a2b4d5db1e7ce04034cc275'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.258014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44490a948a2b4d5db1e7ce04034cc275'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.290288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44490a948a2b4d5db1e7ce04034cc275'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.291200 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44490a948a2b4d5db1e7ce04034cc275'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.311957 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44490a948a2b4d5db1e7ce04034cc275'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.322385 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1047/2109] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:27 2022] DELETE /identity/v3/users/44490a948a2b4d5db1e7ce04034cc275 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:27.328850 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807e54b-8ebe-4981-9029-6ff2e02ddd83 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:27.329375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807e54b-8ebe-4981-9029-6ff2e02ddd83 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:27.329865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807e54b-8ebe-4981-9029-6ff2e02ddd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:27.361322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8807e54b-8ebe-4981-9029-6ff2e02ddd83 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:27.363341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8807e54b-8ebe-4981-9029-6ff2e02ddd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:27.462135 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:27.468167 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:27.469581 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1063/2110] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:27.489199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807e54b-8ebe-4981-9029-6ff2e02ddd83 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:27.490829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8807e54b-8ebe-4981-9029-6ff2e02ddd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:27.497361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0e468-2775-46cd-9d1c-dfdf29aa7cb6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:27.507412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b749db-b2f7-47cc-b90b-9a91ef61e901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:27.508187 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b749db-b2f7-47cc-b90b-9a91ef61e901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:27.508567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b749db-b2f7-47cc-b90b-9a91ef61e901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:27.508940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b749db-b2f7-47cc-b90b-9a91ef61e901 admin admin] PATH_INFO: `/v3/users/1eb9b5b3061e4330b19882b62cc4e5f2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:27.510187 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b749db-b2f7-47cc-b90b-9a91ef61e901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb9b5b3061e4330b19882b62cc4e5f2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:27.512329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b749db-b2f7-47cc-b90b-9a91ef61e901 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:27.573039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807e54b-8ebe-4981-9029-6ff2e02ddd83 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:27.578144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ad89a-f503-431d-a9e3-c4851eb636cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:27.578914 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ad89a-f503-431d-a9e3-c4851eb636cd service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:27.579297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ad89a-f503-431d-a9e3-c4851eb636cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:27.579597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ad89a-f503-431d-a9e3-c4851eb636cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:27.601576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ad89a-f503-431d-a9e3-c4851eb636cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:27.605040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ad89a-f503-431d-a9e3-c4851eb636cd service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:27.630223 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-10b749db-b2f7-47cc-b90b-9a91ef61e901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb9b5b3061e4330b19882b62cc4e5f2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.633232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1064/2111] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:16:27.635545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ad89a-f503-431d-a9e3-c4851eb636cd service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:27.637623 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-311ad89a-f503-431d-a9e3-c4851eb636cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:27.649507 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-10b749db-b2f7-47cc-b90b-9a91ef61e901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb9b5b3061e4330b19882b62cc4e5f2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.660658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ad89a-f503-431d-a9e3-c4851eb636cd service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:27.667487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11610bf-f477-4e70-93cf-0168d709fa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:27.668426 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11610bf-f477-4e70-93cf-0168d709fa1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:27.669552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11610bf-f477-4e70-93cf-0168d709fa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:27.669897 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11610bf-f477-4e70-93cf-0168d709fa1b admin admin] PATH_INFO: `/v3/projects/8cbb8921820b4b5abc78e3d1b9af1a07` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:27.672200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11610bf-f477-4e70-93cf-0168d709fa1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cbb8921820b4b5abc78e3d1b9af1a07)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:27.674622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11610bf-f477-4e70-93cf-0168d709fa1b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:27.676069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-10b749db-b2f7-47cc-b90b-9a91ef61e901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb9b5b3061e4330b19882b62cc4e5f2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.679665 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-10b749db-b2f7-47cc-b90b-9a91ef61e901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb9b5b3061e4330b19882b62cc4e5f2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.694319 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-10b749db-b2f7-47cc-b90b-9a91ef61e901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb9b5b3061e4330b19882b62cc4e5f2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.713835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1048/2112] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:27 2022] DELETE /identity/v3/users/1eb9b5b3061e4330b19882b62cc4e5f2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:27.750290 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:27.766248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a11610bf-f477-4e70-93cf-0168d709fa1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cbb8921820b4b5abc78e3d1b9af1a07', 'request_id': 'req-a11610bf-f477-4e70-93cf-0168d709fa1b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.784106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a11610bf-f477-4e70-93cf-0168d709fa1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cbb8921820b4b5abc78e3d1b9af1a07', 'request_id': 'req-a11610bf-f477-4e70-93cf-0168d709fa1b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.796688 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a11610bf-f477-4e70-93cf-0168d709fa1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cbb8921820b4b5abc78e3d1b9af1a07', 'request_id': 'req-a11610bf-f477-4e70-93cf-0168d709fa1b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.815614 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a11610bf-f477-4e70-93cf-0168d709fa1b admin admin] The token cache is being invalidate because project 8cbb8921820b4b5abc78e3d1b9af1a07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:27.816009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a11610bf-f477-4e70-93cf-0168d709fa1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.816849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a11610bf-f477-4e70-93cf-0168d709fa1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:27.818497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1065/2113] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:27 2022] DELETE /identity/v3/projects/8cbb8921820b4b5abc78e3d1b9af1a07 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:27.827152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c63d0-10ea-4be3-aace-618f19d33bd5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:27.829218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c63d0-10ea-4be3-aace-618f19d33bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:27.829218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c63d0-10ea-4be3-aace-618f19d33bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:27.864790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4b3c63d0-10ea-4be3-aace-618f19d33bd5 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:27.866147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4b3c63d0-10ea-4be3-aace-618f19d33bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:27.872054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:27.960456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1049/2114] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:27.969657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf309108-ae6e-4143-98a8-f0ef1c127e2a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:27.969948 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf309108-ae6e-4143-98a8-f0ef1c127e2a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:27.970221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf309108-ae6e-4143-98a8-f0ef1c127e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:27.993671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-bf309108-ae6e-4143-98a8-f0ef1c127e2a None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:27.995415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bf309108-ae6e-4143-98a8-f0ef1c127e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:28.011105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1066/2115] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:28.013411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c63d0-10ea-4be3-aace-618f19d33bd5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:28.047994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c63d0-10ea-4be3-aace-618f19d33bd5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:28.055458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a3158-c47c-4aea-8992-d1a2c4bcca6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:28.056156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a3158-c47c-4aea-8992-d1a2c4bcca6f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:28.056502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a3158-c47c-4aea-8992-d1a2c4bcca6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:28.056778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a3158-c47c-4aea-8992-d1a2c4bcca6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:28.057708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a3158-c47c-4aea-8992-d1a2c4bcca6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:16:28.058899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a3158-c47c-4aea-8992-d1a2c4bcca6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:28.062637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a3158-c47c-4aea-8992-d1a2c4bcca6f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:28.078292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1050/2116] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:16:28.083480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf309108-ae6e-4143-98a8-f0ef1c127e2a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:28.101490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf309108-ae6e-4143-98a8-f0ef1c127e2a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:28.108980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981dcbd-4053-4759-9697-cc3fe29e762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:28.109623 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981dcbd-4053-4759-9697-cc3fe29e762f admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:28.110413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981dcbd-4053-4759-9697-cc3fe29e762f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:28.110714 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981dcbd-4053-4759-9697-cc3fe29e762f admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:28.111399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981dcbd-4053-4759-9697-cc3fe29e762f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:28.113876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981dcbd-4053-4759-9697-cc3fe29e762f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:28.180815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1067/2117] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:28 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:28.182649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a3158-c47c-4aea-8992-d1a2c4bcca6f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:28.196199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981dcbd-4053-4759-9697-cc3fe29e762f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:28.198357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2981dcbd-4053-4759-9697-cc3fe29e762f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:28.201363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a3158-c47c-4aea-8992-d1a2c4bcca6f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:28.344800 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745a412-3212-44d0-b599-78816c4005e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:28.345429 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745a412-3212-44d0-b599-78816c4005e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:28.345712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745a412-3212-44d0-b599-78816c4005e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:28.345990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745a412-3212-44d0-b599-78816c4005e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:28.346628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745a412-3212-44d0-b599-78816c4005e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:28.350724 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745a412-3212-44d0-b599-78816c4005e3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:28.375566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981dcbd-4053-4759-9697-cc3fe29e762f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:28.385306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:28.386253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:28.386586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:28.387106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] PATH_INFO: `/v3/projects/eb531234364c4f94a994e58c2b861199` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:28.389418 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb531234364c4f94a994e58c2b861199)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:28.396288 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:28.409371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1051/2118] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:28 2022] POST /identity/v3/users => generated 282 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:28.410285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745a412-3212-44d0-b599-78816c4005e3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:28.415544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3745a412-3212-44d0-b599-78816c4005e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:28.441130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745a412-3212-44d0-b599-78816c4005e3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:28.449155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:28.451593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:28.452241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:28.452684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] PATH_INFO: `/v3/projects/3e81d7a7e59c4daf825e7313e0a62763` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:28.454090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e81d7a7e59c4daf825e7313e0a62763)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:28.457201 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:28.513322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb531234364c4f94a994e58c2b861199', 'request_id': 'req-d8e1b371-d295-47f2-a70b-3efc8cea94ba'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.526579 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb531234364c4f94a994e58c2b861199', 'request_id': 'req-d8e1b371-d295-47f2-a70b-3efc8cea94ba'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.539869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb531234364c4f94a994e58c2b861199', 'request_id': 'req-d8e1b371-d295-47f2-a70b-3efc8cea94ba'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.552393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e81d7a7e59c4daf825e7313e0a62763', 'request_id': 'req-13918717-b5b9-4772-a46a-4f1ecaf60dd3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.566822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] The token cache is being invalidate because project eb531234364c4f94a994e58c2b861199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:28.569045 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e81d7a7e59c4daf825e7313e0a62763', 'request_id': 'req-13918717-b5b9-4772-a46a-4f1ecaf60dd3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.569535 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.571763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.577896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1068/2119] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:28 2022] DELETE /identity/v3/projects/eb531234364c4f94a994e58c2b861199 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:28.579019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:28.582711 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:28.583702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e81d7a7e59c4daf825e7313e0a62763', 'request_id': 'req-13918717-b5b9-4772-a46a-4f1ecaf60dd3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.647145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] The token cache is being invalidate because project 3e81d7a7e59c4daf825e7313e0a62763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:28.647445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.648171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.654423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1052/2120] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:28 2022] DELETE /identity/v3/projects/3e81d7a7e59c4daf825e7313e0a62763 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:28.654423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:28.654423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:28.659398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:28.669103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:28.714834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b371-d295-47f2-a70b-3efc8cea94ba admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:28.755467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13918717-b5b9-4772-a46a-4f1ecaf60dd3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:28.760562 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90403baa-b23d-44c6-a646-d5fcb339a333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:28.761408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90403baa-b23d-44c6-a646-d5fcb339a333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:28.761786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90403baa-b23d-44c6-a646-d5fcb339a333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:28.762697 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90403baa-b23d-44c6-a646-d5fcb339a333 admin admin] PATH_INFO: `/v3/users/7cbaeeff37314ec7bfb8cd8e26243e30` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:28.764564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90403baa-b23d-44c6-a646-d5fcb339a333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cbaeeff37314ec7bfb8cd8e26243e30)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:28.766776 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90403baa-b23d-44c6-a646-d5fcb339a333 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:28.803580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f56d59-8c35-44e4-ad50-0af04eff0480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:28.804554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f56d59-8c35-44e4-ad50-0af04eff0480 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:28.804981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f56d59-8c35-44e4-ad50-0af04eff0480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:28.805435 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f56d59-8c35-44e4-ad50-0af04eff0480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:28.806406 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f56d59-8c35-44e4-ad50-0af04eff0480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:28.808380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f56d59-8c35-44e4-ad50-0af04eff0480 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:28.830388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1069/2121] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:28.831992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a837e2f-9639-40fd-b41a-655067b4b268 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:28.832445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a837e2f-9639-40fd-b41a-655067b4b268 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:28.832876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a837e2f-9639-40fd-b41a-655067b4b268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:28.837164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-90403baa-b23d-44c6-a646-d5fcb339a333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cbaeeff37314ec7bfb8cd8e26243e30'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.849800 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-90403baa-b23d-44c6-a646-d5fcb339a333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cbaeeff37314ec7bfb8cd8e26243e30'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.882113 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-90403baa-b23d-44c6-a646-d5fcb339a333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cbaeeff37314ec7bfb8cd8e26243e30'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.883268 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-90403baa-b23d-44c6-a646-d5fcb339a333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cbaeeff37314ec7bfb8cd8e26243e30'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.886298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0a837e2f-9639-40fd-b41a-655067b4b268 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:28.888909 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0a837e2f-9639-40fd-b41a-655067b4b268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:28.904379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-90403baa-b23d-44c6-a646-d5fcb339a333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cbaeeff37314ec7bfb8cd8e26243e30'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:28.910529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1070/2122] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:28.911841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a837e2f-9639-40fd-b41a-655067b4b268 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:28.914153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0a837e2f-9639-40fd-b41a-655067b4b268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:28.919467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1053/2123] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:28 2022] DELETE /identity/v3/users/7cbaeeff37314ec7bfb8cd8e26243e30 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:28.921034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fefce9-6d91-47bc-ba3f-fc25231c424e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:28.922713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fefce9-6d91-47bc-ba3f-fc25231c424e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:28.922713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fefce9-6d91-47bc-ba3f-fc25231c424e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:28.928264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a837e2f-9639-40fd-b41a-655067b4b268 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:28.964632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b4fefce9-6d91-47bc-ba3f-fc25231c424e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:28.967236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b4fefce9-6d91-47bc-ba3f-fc25231c424e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:29.041734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1054/2124] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:29.046780 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fefce9-6d91-47bc-ba3f-fc25231c424e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:29.055003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b82579-8fb1-4456-9254-9ba0b549e2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:29.055841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b82579-8fb1-4456-9254-9ba0b549e2f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:29.056094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b82579-8fb1-4456-9254-9ba0b549e2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:29.056334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b82579-8fb1-4456-9254-9ba0b549e2f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:29.056994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b82579-8fb1-4456-9254-9ba0b549e2f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:29.060941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b82579-8fb1-4456-9254-9ba0b549e2f7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:29.065904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fefce9-6d91-47bc-ba3f-fc25231c424e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:29.073154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad9788-cf41-4372-beff-82a60e4b2485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:29.073154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad9788-cf41-4372-beff-82a60e4b2485 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:29.073154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad9788-cf41-4372-beff-82a60e4b2485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:29.073592 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad9788-cf41-4372-beff-82a60e4b2485 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:29.074259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad9788-cf41-4372-beff-82a60e4b2485 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:16:29.074567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad9788-cf41-4372-beff-82a60e4b2485 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:29.076765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad9788-cf41-4372-beff-82a60e4b2485 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:29.079832 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1071/2125] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:29.082663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b82579-8fb1-4456-9254-9ba0b549e2f7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:29.095096 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1055/2126] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:16:29.096750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad9788-cf41-4372-beff-82a60e4b2485 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:29.105909 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b82579-8fb1-4456-9254-9ba0b549e2f7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:29.108944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad9788-cf41-4372-beff-82a60e4b2485 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:29.114936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39b96a-6779-48d7-83ae-e8d90942ce76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:29.115334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c41f3-71df-4583-b261-24be806daf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:29.116098 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39b96a-6779-48d7-83ae-e8d90942ce76 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:29.116645 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39b96a-6779-48d7-83ae-e8d90942ce76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:29.116919 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39b96a-6779-48d7-83ae-e8d90942ce76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:29.117065 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c41f3-71df-4583-b261-24be806daf46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:29.117476 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c41f3-71df-4583-b261-24be806daf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:29.117896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39b96a-6779-48d7-83ae-e8d90942ce76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:29.119082 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c41f3-71df-4583-b261-24be806daf46 admin admin] PATH_INFO: `/v3/projects/b59568c5604c4bfb9564fff58a162f6d/users/5a0abb18539c4116b48fe697dd3aa82d/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:29.122161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39b96a-6779-48d7-83ae-e8d90942ce76 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:29.142399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c41f3-71df-4583-b261-24be806daf46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b59568c5604c4bfb9564fff58a162f6d, user_id=5a0abb18539c4116b48fe697dd3aa82d, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:29.145326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c41f3-71df-4583-b261-24be806daf46 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:29.147707 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:29.172643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1072/2127] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:29 2022] PUT /identity/v3/projects/b59568c5604c4bfb9564fff58a162f6d/users/5a0abb18539c4116b48fe697dd3aa82d/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:16:29.181143 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1056/2128] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:29 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:16:29.185773 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c41f3-71df-4583-b261-24be806daf46 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:29.187786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b23a3-fbd0-4ec6-b73a-3eb68670b523 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:29.188232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b23a3-fbd0-4ec6-b73a-3eb68670b523 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:29.188504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b23a3-fbd0-4ec6-b73a-3eb68670b523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:29.225288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-938b23a3-fbd0-4ec6-b73a-3eb68670b523 None None] MFA Rules not processed for user `5a0abb18539c4116b48fe697dd3aa82d`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:29.226821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-938b23a3-fbd0-4ec6-b73a-3eb68670b523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:29.250765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:29.280962 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c41f3-71df-4583-b261-24be806daf46 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:29.321948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1057/2129] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:29.332934 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b23a3-fbd0-4ec6-b73a-3eb68670b523 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:29.341411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-938b23a3-fbd0-4ec6-b73a-3eb68670b523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:29.395994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a065a-5dea-4258-886d-848df7c9624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:29.397431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a065a-5dea-4258-886d-848df7c9624c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:29.397431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a065a-5dea-4258-886d-848df7c9624c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:29.397918 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a065a-5dea-4258-886d-848df7c9624c admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:29.398671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a065a-5dea-4258-886d-848df7c9624c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:29.406462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a065a-5dea-4258-886d-848df7c9624c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:29.426468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b23a3-fbd0-4ec6-b73a-3eb68670b523 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:29.431795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab66fe5-0f8f-4e62-82af-dea4bf26594b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:29.432714 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab66fe5-0f8f-4e62-82af-dea4bf26594b service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:29.433064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab66fe5-0f8f-4e62-82af-dea4bf26594b service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:29.433396 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab66fe5-0f8f-4e62-82af-dea4bf26594b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:29.444890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab66fe5-0f8f-4e62-82af-dea4bf26594b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:29.446443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab66fe5-0f8f-4e62-82af-dea4bf26594b service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:29.453526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1073/2130] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:29 2022] POST /identity/v3/users => generated 293 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:29.455203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a065a-5dea-4258-886d-848df7c9624c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:29.459753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1058/2131] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:29.472988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a065a-5dea-4258-886d-848df7c9624c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:29.478811 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d27c90-eee7-47c7-a684-f5dcba846d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:29.479634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d27c90-eee7-47c7-a684-f5dcba846d7f admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:29.479889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d27c90-eee7-47c7-a684-f5dcba846d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:29.480167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d27c90-eee7-47c7-a684-f5dcba846d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:29.480850 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d27c90-eee7-47c7-a684-f5dcba846d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:29.484261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d27c90-eee7-47c7-a684-f5dcba846d7f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:29.502964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1074/2132] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:29.506970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab66fe5-0f8f-4e62-82af-dea4bf26594b service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:29.523987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab66fe5-0f8f-4e62-82af-dea4bf26594b service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:29.530571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd23d-421b-4968-979d-351273cca927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:29.531192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd23d-421b-4968-979d-351273cca927 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:29.531693 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd23d-421b-4968-979d-351273cca927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:29.532094 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd23d-421b-4968-979d-351273cca927 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:29.533078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcd23d-421b-4968-979d-351273cca927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:29.534999 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcd23d-421b-4968-979d-351273cca927 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:29.568599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1059/2133] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:29.572634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d27c90-eee7-47c7-a684-f5dcba846d7f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:29.592037 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d27c90-eee7-47c7-a684-f5dcba846d7f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:29.608062 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19aecc-313b-4182-b0d4-3cc050dece77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:29.608992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19aecc-313b-4182-b0d4-3cc050dece77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:29.609447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19aecc-313b-4182-b0d4-3cc050dece77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:29.610109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19aecc-313b-4182-b0d4-3cc050dece77 admin admin] PATH_INFO: `/v3/projects/606ffec2caa54e2085b63485214ffdb6/users/eaaa1d1bb56b4a89b6c497a820eafe8a/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:29.668603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19aecc-313b-4182-b0d4-3cc050dece77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=606ffec2caa54e2085b63485214ffdb6, user_id=eaaa1d1bb56b4a89b6c497a820eafe8a, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:29.672489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19aecc-313b-4182-b0d4-3cc050dece77 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:29.708510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1075/2134] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:29 2022] PUT /identity/v3/projects/606ffec2caa54e2085b63485214ffdb6/users/eaaa1d1bb56b4a89b6c497a820eafe8a/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:29.711549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9b957-8814-454f-b90a-44b7a049f3a1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:29.711846 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9b957-8814-454f-b90a-44b7a049f3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:29.712129 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9b957-8814-454f-b90a-44b7a049f3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:29.774355 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3ad9b957-8814-454f-b90a-44b7a049f3a1 None None] MFA Rules not processed for user `eaaa1d1bb56b4a89b6c497a820eafe8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:29.777518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3ad9b957-8814-454f-b90a-44b7a049f3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:29.920377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1060/2135] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:29.924950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19aecc-313b-4182-b0d4-3cc050dece77 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:30.081489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19aecc-313b-4182-b0d4-3cc050dece77 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:30.086913 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd7711-68cb-4832-8d33-92179e3f1fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:30.087719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd7711-68cb-4832-8d33-92179e3f1fdc admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:30.088057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd7711-68cb-4832-8d33-92179e3f1fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:30.088334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd7711-68cb-4832-8d33-92179e3f1fdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:30.088983 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd7711-68cb-4832-8d33-92179e3f1fdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:30.091905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd7711-68cb-4832-8d33-92179e3f1fdc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:30.150013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9b957-8814-454f-b90a-44b7a049f3a1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:30.159119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:30.159876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1076/2136] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:29 2022] POST /identity/v3/projects => generated 383 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:30.164835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9b957-8814-454f-b90a-44b7a049f3a1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:30.265809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb507f3-e3c7-43c3-94b0-fcf94ab1ea85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:30.267236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb507f3-e3c7-43c3-94b0-fcf94ab1ea85 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:30.267725 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb507f3-e3c7-43c3-94b0-fcf94ab1ea85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:30.268352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb507f3-e3c7-43c3-94b0-fcf94ab1ea85 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:30.269798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb507f3-e3c7-43c3-94b0-fcf94ab1ea85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:30.273655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb507f3-e3c7-43c3-94b0-fcf94ab1ea85 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:30.292365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd7711-68cb-4832-8d33-92179e3f1fdc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:30.295518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0efd7711-68cb-4832-8d33-92179e3f1fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:30.320290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1061/2137] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:30 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:16:30.323950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb507f3-e3c7-43c3-94b0-fcf94ab1ea85 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:30.336678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd7711-68cb-4832-8d33-92179e3f1fdc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:30.343746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb507f3-e3c7-43c3-94b0-fcf94ab1ea85 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:30.353079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027a892-54b5-446e-ba6c-2ba1b84ebdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:30.353774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027a892-54b5-446e-ba6c-2ba1b84ebdbe admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:30.354156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027a892-54b5-446e-ba6c-2ba1b84ebdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:30.354374 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de277a-743e-40d6-aa4f-983c17af3ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:30.354506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027a892-54b5-446e-ba6c-2ba1b84ebdbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:30.355161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027a892-54b5-446e-ba6c-2ba1b84ebdbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:30.355480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de277a-743e-40d6-aa4f-983c17af3ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:30.355847 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de277a-743e-40d6-aa4f-983c17af3ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:30.356363 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de277a-743e-40d6-aa4f-983c17af3ba6 admin admin] PATH_INFO: `/v3/projects/0a88369caf124bf9ad9b7edf3e145009` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:30.357115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027a892-54b5-446e-ba6c-2ba1b84ebdbe admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:30.358318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de277a-743e-40d6-aa4f-983c17af3ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a88369caf124bf9ad9b7edf3e145009)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:30.360747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de277a-743e-40d6-aa4f-983c17af3ba6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:30.375504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1062/2138] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:16:30.379836 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027a892-54b5-446e-ba6c-2ba1b84ebdbe admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:30.397328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027a892-54b5-446e-ba6c-2ba1b84ebdbe admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:30.404246 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3b6fe-7f44-41cb-9059-deeaacd04a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:30.404945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3b6fe-7f44-41cb-9059-deeaacd04a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:30.405228 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3b6fe-7f44-41cb-9059-deeaacd04a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:30.405558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3b6fe-7f44-41cb-9059-deeaacd04a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:30.406211 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3b6fe-7f44-41cb-9059-deeaacd04a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:30.412396 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3b6fe-7f44-41cb-9059-deeaacd04a0b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:30.417763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:30.453073 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1063/2139] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:16:30.457136 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3b6fe-7f44-41cb-9059-deeaacd04a0b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:30.476984 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3b6fe-7f44-41cb-9059-deeaacd04a0b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:30.478970 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5de277a-743e-40d6-aa4f-983c17af3ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a88369caf124bf9ad9b7edf3e145009', 'request_id': 'req-e5de277a-743e-40d6-aa4f-983c17af3ba6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:30.500528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5de277a-743e-40d6-aa4f-983c17af3ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a88369caf124bf9ad9b7edf3e145009', 'request_id': 'req-e5de277a-743e-40d6-aa4f-983c17af3ba6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:30.508644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5de277a-743e-40d6-aa4f-983c17af3ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a88369caf124bf9ad9b7edf3e145009', 'request_id': 'req-e5de277a-743e-40d6-aa4f-983c17af3ba6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:30.529578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5de277a-743e-40d6-aa4f-983c17af3ba6 admin admin] The token cache is being invalidate because project 0a88369caf124bf9ad9b7edf3e145009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:30.530122 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5de277a-743e-40d6-aa4f-983c17af3ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:30.531663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5de277a-743e-40d6-aa4f-983c17af3ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:30.534346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1077/2140] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:30 2022] DELETE /identity/v3/projects/0a88369caf124bf9ad9b7edf3e145009 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:30.539285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a85d58-acf2-4494-850b-814c6487d4ab None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:30.539816 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a85d58-acf2-4494-850b-814c6487d4ab None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:30.540264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a85d58-acf2-4494-850b-814c6487d4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:30.576507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-62a85d58-acf2-4494-850b-814c6487d4ab None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:30.578802 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-62a85d58-acf2-4494-850b-814c6487d4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:30.597403 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2261e-4286-4c59-a650-73965a62e3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:30.598386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2261e-4286-4c59-a650-73965a62e3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:30.598824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2261e-4286-4c59-a650-73965a62e3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:30.599279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2261e-4286-4c59-a650-73965a62e3f7 admin admin] PATH_INFO: `/v3/projects/e7f6b887fb204970a52afd3fad2e1e7f/users/85c999b0c7184e4997138a20a636f072/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:30.627898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2261e-4286-4c59-a650-73965a62e3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f6b887fb204970a52afd3fad2e1e7f, user_id=85c999b0c7184e4997138a20a636f072, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:30.630924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2261e-4286-4c59-a650-73965a62e3f7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:30.650505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1064/2141] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:30 2022] PUT /identity/v3/projects/e7f6b887fb204970a52afd3fad2e1e7f/users/85c999b0c7184e4997138a20a636f072/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:16:30.652208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a40b35-a170-4f3e-bad6-9c3493620f93 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:30.652685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a40b35-a170-4f3e-bad6-9c3493620f93 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:30.653992 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a40b35-a170-4f3e-bad6-9c3493620f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:30.693210 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a1a40b35-a170-4f3e-bad6-9c3493620f93 None None] MFA Rules not processed for user `ca9eb6b9dc6e409a954c7993ebe5a5c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:30.694601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a1a40b35-a170-4f3e-bad6-9c3493620f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:30.699158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1078/2142] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:16:30.699158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2df293-1213-4d60-8c20-36575cd62a40 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:30.699158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2df293-1213-4d60-8c20-36575cd62a40 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:30.699158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2df293-1213-4d60-8c20-36575cd62a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:30.734299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7a2df293-1213-4d60-8c20-36575cd62a40 None None] MFA Rules not processed for user `85c999b0c7184e4997138a20a636f072`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:30.735963 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7a2df293-1213-4d60-8c20-36575cd62a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:30.754640 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:30.798392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1065/2143] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:30.799644 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b0e34-d547-4d30-b3a5-dd7905fcb8aa None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:30.800350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b0e34-d547-4d30-b3a5-dd7905fcb8aa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:30.800641 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b0e34-d547-4d30-b3a5-dd7905fcb8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:30.822126 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fd8b0e34-d547-4d30-b3a5-dd7905fcb8aa None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:30.823385 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fd8b0e34-d547-4d30-b3a5-dd7905fcb8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:30.850512 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1079/2144] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:30.851696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2df293-1213-4d60-8c20-36575cd62a40 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:30.854434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7a2df293-1213-4d60-8c20-36575cd62a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:30.911729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:30.911967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1066/2145] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:30.912722 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b0e34-d547-4d30-b3a5-dd7905fcb8aa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:30.916902 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fd8b0e34-d547-4d30-b3a5-dd7905fcb8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:30.929813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b0e34-d547-4d30-b3a5-dd7905fcb8aa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:30.934995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc4883-7c2e-4038-8bc7-0cc200e8ce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:30.935637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc4883-7c2e-4038-8bc7-0cc200e8ce64 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:30.935915 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc4883-7c2e-4038-8bc7-0cc200e8ce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:30.936192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc4883-7c2e-4038-8bc7-0cc200e8ce64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:30.936821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc4883-7c2e-4038-8bc7-0cc200e8ce64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:30.938739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc4883-7c2e-4038-8bc7-0cc200e8ce64 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:30.950220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2df293-1213-4d60-8c20-36575cd62a40 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:30.956097 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61940b2f-3ccd-4f5d-a24e-0058bf9c28eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:30.956789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61940b2f-3ccd-4f5d-a24e-0058bf9c28eb service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:30.957142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61940b2f-3ccd-4f5d-a24e-0058bf9c28eb service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:30.957329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61940b2f-3ccd-4f5d-a24e-0058bf9c28eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:30.965751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61940b2f-3ccd-4f5d-a24e-0058bf9c28eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:30.966528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1067/2146] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:30 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:30.967444 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc4883-7c2e-4038-8bc7-0cc200e8ce64 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:30.967706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61940b2f-3ccd-4f5d-a24e-0058bf9c28eb service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:30.976931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1080/2147] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:30.977720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61940b2f-3ccd-4f5d-a24e-0058bf9c28eb service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:30.988361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61940b2f-3ccd-4f5d-a24e-0058bf9c28eb service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:31.039452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc4883-7c2e-4038-8bc7-0cc200e8ce64 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:31.050955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6a8e4-aa75-46dd-aa4c-5f919fef0805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:31.051803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6a8e4-aa75-46dd-aa4c-5f919fef0805 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:31.052087 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6a8e4-aa75-46dd-aa4c-5f919fef0805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:31.052363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6a8e4-aa75-46dd-aa4c-5f919fef0805 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:31.053194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6a8e4-aa75-46dd-aa4c-5f919fef0805 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:16:31.064912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6a8e4-aa75-46dd-aa4c-5f919fef0805 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:31.067084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6a8e4-aa75-46dd-aa4c-5f919fef0805 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:31.091074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557c096-9d05-4809-a490-a075190b7b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:31.091921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557c096-9d05-4809-a490-a075190b7b0c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:31.092577 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557c096-9d05-4809-a490-a075190b7b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:31.093142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557c096-9d05-4809-a490-a075190b7b0c admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:31.093965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557c096-9d05-4809-a490-a075190b7b0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:31.103953 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557c096-9d05-4809-a490-a075190b7b0c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:31.104760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1068/2148] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:16:31.109987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6a8e4-aa75-46dd-aa4c-5f919fef0805 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:31.123429 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6a8e4-aa75-46dd-aa4c-5f919fef0805 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:31.131145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9f04c-436b-4eff-9f53-4e8b2e3fb49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:31.131997 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9f04c-436b-4eff-9f53-4e8b2e3fb49c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:31.132404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9f04c-436b-4eff-9f53-4e8b2e3fb49c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:31.133066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9f04c-436b-4eff-9f53-4e8b2e3fb49c admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:31.133775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9f04c-436b-4eff-9f53-4e8b2e3fb49c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:31.136194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9f04c-436b-4eff-9f53-4e8b2e3fb49c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:31.136326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1081/2149] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:30 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:31.137417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557c096-9d05-4809-a490-a075190b7b0c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:31.149961 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557c096-9d05-4809-a490-a075190b7b0c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:31.160807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333975c1-ce39-4c0a-b634-bcddb8ea4531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:31.161443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333975c1-ce39-4c0a-b634-bcddb8ea4531 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:31.161739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333975c1-ce39-4c0a-b634-bcddb8ea4531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:31.162001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333975c1-ce39-4c0a-b634-bcddb8ea4531 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:31.162664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333975c1-ce39-4c0a-b634-bcddb8ea4531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:31.164319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333975c1-ce39-4c0a-b634-bcddb8ea4531 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:31.177432 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1082/2150] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:31.181501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333975c1-ce39-4c0a-b634-bcddb8ea4531 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:31.193120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333975c1-ce39-4c0a-b634-bcddb8ea4531 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:31.194730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1069/2151] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:31 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:16:31.199685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9f04c-436b-4eff-9f53-4e8b2e3fb49c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:31.310096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36426b7b-e187-41dc-add5-78eefe60f1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:31.310791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36426b7b-e187-41dc-add5-78eefe60f1ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:31.311090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36426b7b-e187-41dc-add5-78eefe60f1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:31.311369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36426b7b-e187-41dc-add5-78eefe60f1ca admin admin] PATH_INFO: `/v3/projects/ed46e8d25d474a5ebec3007cf6ed6de4/users/538c85e1067345a19908e94a43a9d095/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:31.320466 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:31.358429 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36426b7b-e187-41dc-add5-78eefe60f1ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed46e8d25d474a5ebec3007cf6ed6de4, user_id=538c85e1067345a19908e94a43a9d095, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:31.363742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36426b7b-e187-41dc-add5-78eefe60f1ca admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:31.384616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9f04c-436b-4eff-9f53-4e8b2e3fb49c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:31.395190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1083/2152] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:31 2022] PUT /identity/v3/projects/ed46e8d25d474a5ebec3007cf6ed6de4/users/538c85e1067345a19908e94a43a9d095/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:31.399453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36426b7b-e187-41dc-add5-78eefe60f1ca admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:31.493613 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:31.561057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36426b7b-e187-41dc-add5-78eefe60f1ca admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:31.575472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437fa1a-cdea-4789-b6d1-32d14b2ab5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:31.576695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437fa1a-cdea-4789-b6d1-32d14b2ab5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:31.577137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437fa1a-cdea-4789-b6d1-32d14b2ab5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:31.577639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437fa1a-cdea-4789-b6d1-32d14b2ab5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:31.585796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437fa1a-cdea-4789-b6d1-32d14b2ab5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:31.589083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437fa1a-cdea-4789-b6d1-32d14b2ab5c4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:31.597350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faa735-f8b9-4ec5-a21e-f75747726322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:31.598262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa735-f8b9-4ec5-a21e-f75747726322 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:31.599019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa735-f8b9-4ec5-a21e-f75747726322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:31.599643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa735-f8b9-4ec5-a21e-f75747726322 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:31.600499 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faa735-f8b9-4ec5-a21e-f75747726322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:31.603431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faa735-f8b9-4ec5-a21e-f75747726322 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:31.610671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1084/2153] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:31.625072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437fa1a-cdea-4789-b6d1-32d14b2ab5c4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:31.647149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437fa1a-cdea-4789-b6d1-32d14b2ab5c4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:31.653493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1070/2154] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:31 2022] POST /identity/v3/users => generated 287 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:16:31.655351 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faa735-f8b9-4ec5-a21e-f75747726322 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:31.659321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e10559-f129-4da2-bfa5-544ceca8e183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:31.660010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e10559-f129-4da2-bfa5-544ceca8e183 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:31.660299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e10559-f129-4da2-bfa5-544ceca8e183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:31.660731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e10559-f129-4da2-bfa5-544ceca8e183 admin admin] PATH_INFO: `/v3/projects/ed46e8d25d474a5ebec3007cf6ed6de4/users/538c85e1067345a19908e94a43a9d095/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:31.664708 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e10559-f129-4da2-bfa5-544ceca8e183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed46e8d25d474a5ebec3007cf6ed6de4, user_id=538c85e1067345a19908e94a43a9d095, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:31.667425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e10559-f129-4da2-bfa5-544ceca8e183 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:31.681307 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faa735-f8b9-4ec5-a21e-f75747726322 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:31.692667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1085/2155] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:31 2022] PUT /identity/v3/projects/ed46e8d25d474a5ebec3007cf6ed6de4/users/538c85e1067345a19908e94a43a9d095/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:31.705684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc4799-5362-4753-9af1-849759251ec8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:31.706361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc4799-5362-4753-9af1-849759251ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:31.706361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc4799-5362-4753-9af1-849759251ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:31.745152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7bdc4799-5362-4753-9af1-849759251ec8 None None] MFA Rules not processed for user `538c85e1067345a19908e94a43a9d095`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:31.746524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7bdc4799-5362-4753-9af1-849759251ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:31.775523 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01c607-a055-4d1f-8f0f-8efc84e671ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:31.776287 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01c607-a055-4d1f-8f0f-8efc84e671ea admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:31.776636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01c607-a055-4d1f-8f0f-8efc84e671ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:31.777006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01c607-a055-4d1f-8f0f-8efc84e671ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:31.778038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01c607-a055-4d1f-8f0f-8efc84e671ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:31.780823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01c607-a055-4d1f-8f0f-8efc84e671ea admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:31.803101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1071/2156] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:16:31.807509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01c607-a055-4d1f-8f0f-8efc84e671ea admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:31.825171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01c607-a055-4d1f-8f0f-8efc84e671ea admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:31.836942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6818c45-677f-465e-a93c-3b66a3a07d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:31.837612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6818c45-677f-465e-a93c-3b66a3a07d25 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:31.838339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6818c45-677f-465e-a93c-3b66a3a07d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:31.838339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6818c45-677f-465e-a93c-3b66a3a07d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:31.838979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6818c45-677f-465e-a93c-3b66a3a07d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:31.841203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6818c45-677f-465e-a93c-3b66a3a07d25 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:31.866181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1072/2157] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:31.871107 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6818c45-677f-465e-a93c-3b66a3a07d25 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:31.871959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1086/2158] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:31.894385 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6818c45-677f-465e-a93c-3b66a3a07d25 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:31.901992 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb2323-004b-4dda-a20c-b127583c2dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:31.903928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb2323-004b-4dda-a20c-b127583c2dfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:31.904233 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb2323-004b-4dda-a20c-b127583c2dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:31.904676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb2323-004b-4dda-a20c-b127583c2dfb admin admin] PATH_INFO: `/v3/projects/26b477f5c3904c99815e142d7b6de92a/users/49ada688a7e148be8dac29606ebfbcd8/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:31.925324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb2323-004b-4dda-a20c-b127583c2dfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b477f5c3904c99815e142d7b6de92a, user_id=49ada688a7e148be8dac29606ebfbcd8, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:31.928763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb2323-004b-4dda-a20c-b127583c2dfb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:31.955708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1073/2159] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:31 2022] PUT /identity/v3/projects/26b477f5c3904c99815e142d7b6de92a/users/49ada688a7e148be8dac29606ebfbcd8/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:31.980288 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa870b2a-9016-408f-897b-11ab2c69a78f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:31.980288 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa870b2a-9016-408f-897b-11ab2c69a78f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:31.980288 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa870b2a-9016-408f-897b-11ab2c69a78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:31.985031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb2323-004b-4dda-a20c-b127583c2dfb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:31.987247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b5fb2323-004b-4dda-a20c-b127583c2dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:32.084210 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:32.098511 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-aa870b2a-9016-408f-897b-11ab2c69a78f None None] MFA Rules not processed for user `49ada688a7e148be8dac29606ebfbcd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:32.103208 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aa870b2a-9016-408f-897b-11ab2c69a78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:32.127962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb2323-004b-4dda-a20c-b127583c2dfb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:32.138189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161bf5b-22a4-4235-93bc-cc37b3d143c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:32.140118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161bf5b-22a4-4235-93bc-cc37b3d143c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:32.140508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161bf5b-22a4-4235-93bc-cc37b3d143c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:32.140893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161bf5b-22a4-4235-93bc-cc37b3d143c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:32.143727 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4161bf5b-22a4-4235-93bc-cc37b3d143c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:32.227767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1087/2160] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:32.244129 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161bf5b-22a4-4235-93bc-cc37b3d143c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:32.259843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161bf5b-22a4-4235-93bc-cc37b3d143c1 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:32.286691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1074/2161] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:16:32.469361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa870b2a-9016-408f-897b-11ab2c69a78f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:32.471784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aa870b2a-9016-408f-897b-11ab2c69a78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:32.519422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa870b2a-9016-408f-897b-11ab2c69a78f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:32.530472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe88a15-f743-4fca-99e0-f3a5bb7daaf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:32.532151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe88a15-f743-4fca-99e0-f3a5bb7daaf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:32.532427 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe88a15-f743-4fca-99e0-f3a5bb7daaf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:32.532695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe88a15-f743-4fca-99e0-f3a5bb7daaf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:32.556512 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe88a15-f743-4fca-99e0-f3a5bb7daaf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:32.557939 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe88a15-f743-4fca-99e0-f3a5bb7daaf8 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:32.585467 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1088/2162] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:33.753694 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62f98c-7385-442e-9832-1c089d7b1356 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:33.754218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62f98c-7385-442e-9832-1c089d7b1356 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:33.754328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62f98c-7385-442e-9832-1c089d7b1356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:33.774462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9b62f98c-7385-442e-9832-1c089d7b1356 None None] MFA Rules not processed for user `ca9eb6b9dc6e409a954c7993ebe5a5c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:33.775843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9b62f98c-7385-442e-9832-1c089d7b1356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:33.836694 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1075/2163] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:34.270814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe88a15-f743-4fca-99e0-f3a5bb7daaf8 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:34.281789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe88a15-f743-4fca-99e0-f3a5bb7daaf8 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:34.285695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb24c9c-55b3-4f44-b19a-cf1c87cefa6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:34.286316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb24c9c-55b3-4f44-b19a-cf1c87cefa6b service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:34.286598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb24c9c-55b3-4f44-b19a-cf1c87cefa6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:34.286859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb24c9c-55b3-4f44-b19a-cf1c87cefa6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:34.308418 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:34.396511 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb24c9c-55b3-4f44-b19a-cf1c87cefa6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:34.399147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb24c9c-55b3-4f44-b19a-cf1c87cefa6b service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:34.421085 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1089/2164] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:35.415344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62f98c-7385-442e-9832-1c089d7b1356 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:35.429970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62f98c-7385-442e-9832-1c089d7b1356 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:35.434334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a37bee-9ce6-4239-9fa3-f2b00c33a585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:35.434839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a37bee-9ce6-4239-9fa3-f2b00c33a585 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:35.435394 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a37bee-9ce6-4239-9fa3-f2b00c33a585 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:35.435962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a37bee-9ce6-4239-9fa3-f2b00c33a585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:35.595184 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a37bee-9ce6-4239-9fa3-f2b00c33a585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:35.597546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a37bee-9ce6-4239-9fa3-f2b00c33a585 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:35.618945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1076/2165] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:35 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:36.783170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb24c9c-55b3-4f44-b19a-cf1c87cefa6b service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:36.870539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb24c9c-55b3-4f44-b19a-cf1c87cefa6b service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:36.876087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:36.876739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:36.877025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:36.877283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] PATH_INFO: `/v3/users/49ada688a7e148be8dac29606ebfbcd8` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:36.878394 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49ada688a7e148be8dac29606ebfbcd8)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:36.880321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:36.935336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49ada688a7e148be8dac29606ebfbcd8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:36.948328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49ada688a7e148be8dac29606ebfbcd8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:36.950061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49ada688a7e148be8dac29606ebfbcd8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:36.968582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49ada688a7e148be8dac29606ebfbcd8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:36.976472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49ada688a7e148be8dac29606ebfbcd8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:36.983347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1090/2166] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:36 2022] DELETE /identity/v3/users/49ada688a7e148be8dac29606ebfbcd8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:36.989025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b0ec2-ca7f-4862-894f-485d47103a77 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:36.989480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b0ec2-ca7f-4862-894f-485d47103a77 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:36.990242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b0ec2-ca7f-4862-894f-485d47103a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:37.014339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c30b0ec2-ca7f-4862-894f-485d47103a77 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:37.015890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c30b0ec2-ca7f-4862-894f-485d47103a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:37.044593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:37.117046 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1077/2167] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:37.118569 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:37.120473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:37.187923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:37.228032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b966b-a0db-4968-bd98-551e474936e2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:37.242749 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0801ebe-15da-4407-9fdb-32da7573aefd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:37.243558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801ebe-15da-4407-9fdb-32da7573aefd service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:37.243941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801ebe-15da-4407-9fdb-32da7573aefd service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:37.244952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801ebe-15da-4407-9fdb-32da7573aefd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:37.259518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0801ebe-15da-4407-9fdb-32da7573aefd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:37.261866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0801ebe-15da-4407-9fdb-32da7573aefd service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:37.278294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1091/2168] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:37.742728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b0ec2-ca7f-4862-894f-485d47103a77 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:37.744447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c30b0ec2-ca7f-4862-894f-485d47103a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:37.791155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b0ec2-ca7f-4862-894f-485d47103a77 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:37.807980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef939e-8a49-4326-9b77-644d216462a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:37.808652 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef939e-8a49-4326-9b77-644d216462a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:37.809455 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef939e-8a49-4326-9b77-644d216462a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:37.820969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef939e-8a49-4326-9b77-644d216462a9 admin admin] PATH_INFO: `/v3/projects/26b477f5c3904c99815e142d7b6de92a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:37.823150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef939e-8a49-4326-9b77-644d216462a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26b477f5c3904c99815e142d7b6de92a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:37.826745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef939e-8a49-4326-9b77-644d216462a9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:37.956182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4eef939e-8a49-4326-9b77-644d216462a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26b477f5c3904c99815e142d7b6de92a', 'request_id': 'req-4eef939e-8a49-4326-9b77-644d216462a9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:37.963627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4eef939e-8a49-4326-9b77-644d216462a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26b477f5c3904c99815e142d7b6de92a', 'request_id': 'req-4eef939e-8a49-4326-9b77-644d216462a9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:37.978771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4eef939e-8a49-4326-9b77-644d216462a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26b477f5c3904c99815e142d7b6de92a', 'request_id': 'req-4eef939e-8a49-4326-9b77-644d216462a9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:38.000127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4eef939e-8a49-4326-9b77-644d216462a9 admin admin] The token cache is being invalidate because project 26b477f5c3904c99815e142d7b6de92a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:38.003407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4eef939e-8a49-4326-9b77-644d216462a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:38.016398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4eef939e-8a49-4326-9b77-644d216462a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:38.019625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1078/2169] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:37 2022] DELETE /identity/v3/projects/26b477f5c3904c99815e142d7b6de92a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:38.024423 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458d301-4ee4-49c5-b8dd-eb86e3097bdb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:38.024888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458d301-4ee4-49c5-b8dd-eb86e3097bdb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:38.025793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458d301-4ee4-49c5-b8dd-eb86e3097bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:38.078976 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1458d301-4ee4-49c5-b8dd-eb86e3097bdb None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:38.080590 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1458d301-4ee4-49c5-b8dd-eb86e3097bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:38.231414 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1092/2170] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:38.244601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc623e7-a9e1-4f7e-8360-1e294768962c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:38.244893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc623e7-a9e1-4f7e-8360-1e294768962c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:38.245163 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc623e7-a9e1-4f7e-8360-1e294768962c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:38.271151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2dc623e7-a9e1-4f7e-8360-1e294768962c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:38.272728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2dc623e7-a9e1-4f7e-8360-1e294768962c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:38.284755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1079/2171] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:38.288884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458d301-4ee4-49c5-b8dd-eb86e3097bdb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:38.301147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458d301-4ee4-49c5-b8dd-eb86e3097bdb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:38.311305 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2467307-45fb-4654-8a32-35a297d9a0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:38.312054 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2467307-45fb-4654-8a32-35a297d9a0bf admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:38.312472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2467307-45fb-4654-8a32-35a297d9a0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:38.312754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2467307-45fb-4654-8a32-35a297d9a0bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:38.313476 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2467307-45fb-4654-8a32-35a297d9a0bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:16:38.313788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2467307-45fb-4654-8a32-35a297d9a0bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:38.315527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2467307-45fb-4654-8a32-35a297d9a0bf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:38.335158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1093/2172] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:16:38.338840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc623e7-a9e1-4f7e-8360-1e294768962c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:38.391344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc623e7-a9e1-4f7e-8360-1e294768962c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:38.405550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca3f0e-5c80-4a00-ba71-5ec20db44787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:38.406531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca3f0e-5c80-4a00-ba71-5ec20db44787 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:38.406949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca3f0e-5c80-4a00-ba71-5ec20db44787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:38.407435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca3f0e-5c80-4a00-ba71-5ec20db44787 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:38.408639 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca3f0e-5c80-4a00-ba71-5ec20db44787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:38.411625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca3f0e-5c80-4a00-ba71-5ec20db44787 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:38.456678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1080/2173] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:38 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:16:38.460873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2467307-45fb-4654-8a32-35a297d9a0bf admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:38.485999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2467307-45fb-4654-8a32-35a297d9a0bf admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:38.568985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd25ae6-922d-4b12-89bc-c8fd228cb44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:38.569696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd25ae6-922d-4b12-89bc-c8fd228cb44b admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:38.570041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd25ae6-922d-4b12-89bc-c8fd228cb44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:38.570339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd25ae6-922d-4b12-89bc-c8fd228cb44b admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:38.571096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd25ae6-922d-4b12-89bc-c8fd228cb44b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:38.573182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd25ae6-922d-4b12-89bc-c8fd228cb44b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:38.646743 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1094/2174] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:38 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:38.647903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca3f0e-5c80-4a00-ba71-5ec20db44787 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:38.670367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca3f0e-5c80-4a00-ba71-5ec20db44787 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:38.676709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a1db9-d014-4e3e-9419-c1ee90aa8f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:38.677720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a1db9-d014-4e3e-9419-c1ee90aa8f0f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:38.678179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a1db9-d014-4e3e-9419-c1ee90aa8f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:38.678654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a1db9-d014-4e3e-9419-c1ee90aa8f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:38.679522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a1db9-d014-4e3e-9419-c1ee90aa8f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:38.681833 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a1db9-d014-4e3e-9419-c1ee90aa8f0f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:38.712571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1081/2175] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:38.713897 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd25ae6-922d-4b12-89bc-c8fd228cb44b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:38.732159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd25ae6-922d-4b12-89bc-c8fd228cb44b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:38.760550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d2daf-1ebd-4835-8302-8f3d4c5f6502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:38.761190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d2daf-1ebd-4835-8302-8f3d4c5f6502 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:38.761473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d2daf-1ebd-4835-8302-8f3d4c5f6502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:38.761749 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d2daf-1ebd-4835-8302-8f3d4c5f6502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:38.762528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d2daf-1ebd-4835-8302-8f3d4c5f6502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:38.764991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d2daf-1ebd-4835-8302-8f3d4c5f6502 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:38.790268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1095/2176] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:38.792116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a1db9-d014-4e3e-9419-c1ee90aa8f0f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:38.805587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a1db9-d014-4e3e-9419-c1ee90aa8f0f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:38.813600 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13db7fd-71ce-4f58-8487-a30140115f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:38.814294 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13db7fd-71ce-4f58-8487-a30140115f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:38.814799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13db7fd-71ce-4f58-8487-a30140115f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:38.815112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13db7fd-71ce-4f58-8487-a30140115f8f admin admin] PATH_INFO: `/v3/projects/3586ecca9d18472491fe46f6da0dcf0a/users/c03bbf274f3f477b9d5308d44b49ca54/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:38.843879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13db7fd-71ce-4f58-8487-a30140115f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3586ecca9d18472491fe46f6da0dcf0a, user_id=c03bbf274f3f477b9d5308d44b49ca54, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:38.856683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13db7fd-71ce-4f58-8487-a30140115f8f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:38.881052 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1082/2177] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:38 2022] PUT /identity/v3/projects/3586ecca9d18472491fe46f6da0dcf0a/users/c03bbf274f3f477b9d5308d44b49ca54/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:16:38.892240 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6234010-b875-4f94-a358-8d83b4e1de70 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:38.893068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6234010-b875-4f94-a358-8d83b4e1de70 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:38.893280 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6234010-b875-4f94-a358-8d83b4e1de70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:38.974562 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d6234010-b875-4f94-a358-8d83b4e1de70 None None] MFA Rules not processed for user `c03bbf274f3f477b9d5308d44b49ca54`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:38.977847 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d6234010-b875-4f94-a358-8d83b4e1de70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:39.018896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:39.092381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1096/2178] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:39.096610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13db7fd-71ce-4f58-8487-a30140115f8f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:39.157335 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:39.178054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13db7fd-71ce-4f58-8487-a30140115f8f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:39.185132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7494a-ad4c-4175-bfdd-5a11e422fe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:39.186049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7494a-ad4c-4175-bfdd-5a11e422fe90 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:39.186889 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7494a-ad4c-4175-bfdd-5a11e422fe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:39.187388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7494a-ad4c-4175-bfdd-5a11e422fe90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:39.190005 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7494a-ad4c-4175-bfdd-5a11e422fe90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:39.192226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7494a-ad4c-4175-bfdd-5a11e422fe90 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:39.233100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1083/2179] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:39 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:39.237312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6234010-b875-4f94-a358-8d83b4e1de70 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:39.251229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6234010-b875-4f94-a358-8d83b4e1de70 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:39.322722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb06222-9994-44e9-ad44-a74b349425d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:39.323369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb06222-9994-44e9-ad44-a74b349425d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:39.323652 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb06222-9994-44e9-ad44-a74b349425d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:39.323926 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb06222-9994-44e9-ad44-a74b349425d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:39.324597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb06222-9994-44e9-ad44-a74b349425d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:39.327075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb06222-9994-44e9-ad44-a74b349425d9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:39.352156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1097/2180] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:39 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:39.353314 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7494a-ad4c-4175-bfdd-5a11e422fe90 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:39.363689 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7494a-ad4c-4175-bfdd-5a11e422fe90 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:39.368522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5d18e-d3d5-4bc3-bb15-92eb78163164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:39.369115 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5d18e-d3d5-4bc3-bb15-92eb78163164 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:39.369393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5d18e-d3d5-4bc3-bb15-92eb78163164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:39.369666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5d18e-d3d5-4bc3-bb15-92eb78163164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:39.370352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe5d18e-d3d5-4bc3-bb15-92eb78163164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:39.372508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe5d18e-d3d5-4bc3-bb15-92eb78163164 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:39.382885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1084/2181] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:39.386678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb06222-9994-44e9-ad44-a74b349425d9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:39.397251 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb06222-9994-44e9-ad44-a74b349425d9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:39.401882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebee91e-5edf-48d1-b688-65507cf76346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:39.402468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebee91e-5edf-48d1-b688-65507cf76346 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:39.402822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebee91e-5edf-48d1-b688-65507cf76346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:39.403329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebee91e-5edf-48d1-b688-65507cf76346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:39.403967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebee91e-5edf-48d1-b688-65507cf76346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:39.405543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebee91e-5edf-48d1-b688-65507cf76346 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:39.415413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1098/2182] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:16:39.419627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5d18e-d3d5-4bc3-bb15-92eb78163164 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:39.429320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5d18e-d3d5-4bc3-bb15-92eb78163164 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:39.434453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98bb19-456f-4e6a-b795-0162e0f1fa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:39.435234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98bb19-456f-4e6a-b795-0162e0f1fa5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:39.435511 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98bb19-456f-4e6a-b795-0162e0f1fa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:39.435781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98bb19-456f-4e6a-b795-0162e0f1fa5e admin admin] PATH_INFO: `/v3/projects/6313571b417a4798908db239cab1fa7e/users/af8c7a5a97a94fa9b2dfdabfde169039/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:39.451679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98bb19-456f-4e6a-b795-0162e0f1fa5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6313571b417a4798908db239cab1fa7e, user_id=af8c7a5a97a94fa9b2dfdabfde169039, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:39.454633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98bb19-456f-4e6a-b795-0162e0f1fa5e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:39.468515 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1085/2183] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:39 2022] PUT /identity/v3/projects/6313571b417a4798908db239cab1fa7e/users/af8c7a5a97a94fa9b2dfdabfde169039/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:16:39.479609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dc9bd-2950-47b5-88d2-77657c70ad12 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:39.479896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dc9bd-2950-47b5-88d2-77657c70ad12 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:39.480169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dc9bd-2950-47b5-88d2-77657c70ad12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:39.509429 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-487dc9bd-2950-47b5-88d2-77657c70ad12 None None] MFA Rules not processed for user `af8c7a5a97a94fa9b2dfdabfde169039`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:39.510952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-487dc9bd-2950-47b5-88d2-77657c70ad12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:39.579135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1099/2184] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:39.583593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98bb19-456f-4e6a-b795-0162e0f1fa5e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:39.645917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98bb19-456f-4e6a-b795-0162e0f1fa5e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:39.650654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce089591-0b4c-40d9-a34c-bd8d2d572c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:39.651302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce089591-0b4c-40d9-a34c-bd8d2d572c72 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:39.651579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce089591-0b4c-40d9-a34c-bd8d2d572c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:39.651856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce089591-0b4c-40d9-a34c-bd8d2d572c72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:39.652498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce089591-0b4c-40d9-a34c-bd8d2d572c72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:39.654474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce089591-0b4c-40d9-a34c-bd8d2d572c72 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:39.681270 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1086/2185] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:39 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:39.686174 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dc9bd-2950-47b5-88d2-77657c70ad12 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:39.696838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dc9bd-2950-47b5-88d2-77657c70ad12 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:39.729808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:39.756375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42244b50-ffa0-4aac-a7b2-7b73b464600e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:39.756995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42244b50-ffa0-4aac-a7b2-7b73b464600e admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:39.757275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42244b50-ffa0-4aac-a7b2-7b73b464600e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:39.757543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42244b50-ffa0-4aac-a7b2-7b73b464600e admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:39.758188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42244b50-ffa0-4aac-a7b2-7b73b464600e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:39.760134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42244b50-ffa0-4aac-a7b2-7b73b464600e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:39.782239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1100/2186] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:39 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:39.786306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce089591-0b4c-40d9-a34c-bd8d2d572c72 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:39.797102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce089591-0b4c-40d9-a34c-bd8d2d572c72 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:39.801760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93d37e-82eb-43d9-b972-492bb9e13e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:39.802363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93d37e-82eb-43d9-b972-492bb9e13e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:39.802642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93d37e-82eb-43d9-b972-492bb9e13e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:39.802913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93d37e-82eb-43d9-b972-492bb9e13e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:39.803586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93d37e-82eb-43d9-b972-492bb9e13e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:39.805196 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93d37e-82eb-43d9-b972-492bb9e13e29 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:39.816455 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1087/2187] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:39.821052 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42244b50-ffa0-4aac-a7b2-7b73b464600e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:39.832225 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42244b50-ffa0-4aac-a7b2-7b73b464600e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:39.839984 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fb7fe-3ca9-426c-b147-0141e19212f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:39.841344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fb7fe-3ca9-426c-b147-0141e19212f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:39.841623 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fb7fe-3ca9-426c-b147-0141e19212f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:39.841895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fb7fe-3ca9-426c-b147-0141e19212f7 admin admin] PATH_INFO: `/v3/projects/8dfb298e3252405bae02d6921705db20/users/4a2fa9a5b39e4e2d9978eeb9a064877a/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:39.857145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fb7fe-3ca9-426c-b147-0141e19212f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dfb298e3252405bae02d6921705db20, user_id=4a2fa9a5b39e4e2d9978eeb9a064877a, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:39.860213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fb7fe-3ca9-426c-b147-0141e19212f7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:39.877846 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1101/2188] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:39 2022] PUT /identity/v3/projects/8dfb298e3252405bae02d6921705db20/users/4a2fa9a5b39e4e2d9978eeb9a064877a/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:39.886354 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93d37e-82eb-43d9-b972-492bb9e13e29 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:39.897887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93d37e-82eb-43d9-b972-492bb9e13e29 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:39.962731 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f60a4-cca5-4e9c-b863-f34da073ae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:39.963361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f60a4-cca5-4e9c-b863-f34da073ae0b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:39.963696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f60a4-cca5-4e9c-b863-f34da073ae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:39.963911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f60a4-cca5-4e9c-b863-f34da073ae0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:39.964547 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f60a4-cca5-4e9c-b863-f34da073ae0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:39.967303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f60a4-cca5-4e9c-b863-f34da073ae0b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:39.982090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1088/2189] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:39.986851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fb7fe-3ca9-426c-b147-0141e19212f7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:39.998054 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fb7fe-3ca9-426c-b147-0141e19212f7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:40.002901 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0623e-a904-4e3e-9ce9-8104333a2970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:40.003604 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0623e-a904-4e3e-9ce9-8104333a2970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:40.003886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0623e-a904-4e3e-9ce9-8104333a2970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:40.004156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0623e-a904-4e3e-9ce9-8104333a2970 admin admin] PATH_INFO: `/v3/projects/8dfb298e3252405bae02d6921705db20/users/4a2fa9a5b39e4e2d9978eeb9a064877a/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:40.006092 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0623e-a904-4e3e-9ce9-8104333a2970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dfb298e3252405bae02d6921705db20, user_id=4a2fa9a5b39e4e2d9978eeb9a064877a, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:40.008797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0623e-a904-4e3e-9ce9-8104333a2970 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:40.022545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1102/2190] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:39 2022] PUT /identity/v3/projects/8dfb298e3252405bae02d6921705db20/users/4a2fa9a5b39e4e2d9978eeb9a064877a/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:40.037408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e0b86-4d9c-4366-95bc-e7c4013bb6d1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:40.037715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e0b86-4d9c-4366-95bc-e7c4013bb6d1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:40.037988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e0b86-4d9c-4366-95bc-e7c4013bb6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:40.054696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:40.071475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-503e0b86-4d9c-4366-95bc-e7c4013bb6d1 None None] MFA Rules not processed for user `4a2fa9a5b39e4e2d9978eeb9a064877a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:40.072697 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-503e0b86-4d9c-4366-95bc-e7c4013bb6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:40.146498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1089/2191] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:16:40.157975 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0623e-a904-4e3e-9ce9-8104333a2970 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:40.159701 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-36a0623e-a904-4e3e-9ce9-8104333a2970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:40.217699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0623e-a904-4e3e-9ce9-8104333a2970 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:40.222133 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f58aa3-7abf-43e0-b03c-b7bfeec10bd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:40.223146 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f58aa3-7abf-43e0-b03c-b7bfeec10bd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:40.223425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f58aa3-7abf-43e0-b03c-b7bfeec10bd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:40.223688 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f58aa3-7abf-43e0-b03c-b7bfeec10bd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:40.283524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f58aa3-7abf-43e0-b03c-b7bfeec10bd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:40.285032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f58aa3-7abf-43e0-b03c-b7bfeec10bd5 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:40.295037 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1103/2192] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:46.243589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4c0cc-2f1a-41fd-91a3-06ba3007cad9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:46.244101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4c0cc-2f1a-41fd-91a3-06ba3007cad9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:46.244256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4c0cc-2f1a-41fd-91a3-06ba3007cad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:46.267621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bad4c0cc-2f1a-41fd-91a3-06ba3007cad9 None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:46.268745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bad4c0cc-2f1a-41fd-91a3-06ba3007cad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:46.271412 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f58aa3-7abf-43e0-b03c-b7bfeec10bd5 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:46.272894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-69f58aa3-7abf-43e0-b03c-b7bfeec10bd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:46.289217 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f58aa3-7abf-43e0-b03c-b7bfeec10bd5 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:46.293550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b2e91-f498-4a6d-8dd2-840e6f508cdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:46.294713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b2e91-f498-4a6d-8dd2-840e6f508cdb service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:46.295198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b2e91-f498-4a6d-8dd2-840e6f508cdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:46.295759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b2e91-f498-4a6d-8dd2-840e6f508cdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:46.376213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:46.409623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1090/2193] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:16:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:46.412115 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b2e91-f498-4a6d-8dd2-840e6f508cdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:46.413786 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b2e91-f498-4a6d-8dd2-840e6f508cdb service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:46.446980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1104/2194] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:16:46.494675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4c0cc-2f1a-41fd-91a3-06ba3007cad9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:46.496161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bad4c0cc-2f1a-41fd-91a3-06ba3007cad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:46.503921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b2e91-f498-4a6d-8dd2-840e6f508cdb service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:46.505047 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-335b2e91-f498-4a6d-8dd2-840e6f508cdb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:46.587776 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4c0cc-2f1a-41fd-91a3-06ba3007cad9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:46.599431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c1241-7a05-4bf6-8149-ba2c657376f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:46.602843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c1241-7a05-4bf6-8149-ba2c657376f8 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:46.603199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c1241-7a05-4bf6-8149-ba2c657376f8 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:46.603481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c1241-7a05-4bf6-8149-ba2c657376f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:46.620533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b2e91-f498-4a6d-8dd2-840e6f508cdb service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:46.621952 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c1241-7a05-4bf6-8149-ba2c657376f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:46.623856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c1241-7a05-4bf6-8149-ba2c657376f8 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:46.628158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:46.629352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:46.629752 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:46.630279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] PATH_INFO: `/v3/users/eaaa1d1bb56b4a89b6c497a820eafe8a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:46.632204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaaa1d1bb56b4a89b6c497a820eafe8a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:46.634537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:46.636418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1091/2195] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:16:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:16:46.713618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaaa1d1bb56b4a89b6c497a820eafe8a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:46.732499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaaa1d1bb56b4a89b6c497a820eafe8a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:46.732499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaaa1d1bb56b4a89b6c497a820eafe8a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:46.761577 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaaa1d1bb56b4a89b6c497a820eafe8a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:46.776000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaaa1d1bb56b4a89b6c497a820eafe8a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:46.790093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1105/2196] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:46 2022] DELETE /identity/v3/users/eaaa1d1bb56b4a89b6c497a820eafe8a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:16:46.795298 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7242e1-7dfa-41d2-8846-77f81a8da56e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:46.795745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7242e1-7dfa-41d2-8846-77f81a8da56e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:46.796099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7242e1-7dfa-41d2-8846-77f81a8da56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:46.826237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1b7242e1-7dfa-41d2-8846-77f81a8da56e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:46.827876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1b7242e1-7dfa-41d2-8846-77f81a8da56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:46.864500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:46.926238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1092/2197] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:16:46.937314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:46.939496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:47.065976 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b8b8b-4f51-4191-a0bf-a152ed252e2b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:47.072893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77553457-8935-4635-8201-5be91c2d6f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:47.073894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77553457-8935-4635-8201-5be91c2d6f04 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:47.075516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77553457-8935-4635-8201-5be91c2d6f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:47.075943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77553457-8935-4635-8201-5be91c2d6f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:47.092219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77553457-8935-4635-8201-5be91c2d6f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:47.095015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77553457-8935-4635-8201-5be91c2d6f04 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:47.118729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1106/2198] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:16:47.507966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7242e1-7dfa-41d2-8846-77f81a8da56e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:47.509990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1b7242e1-7dfa-41d2-8846-77f81a8da56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:47.601585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7242e1-7dfa-41d2-8846-77f81a8da56e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:47.606308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2088e1-9936-459c-b273-f591d9169b2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:47.607180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2088e1-9936-459c-b273-f591d9169b2c service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:47.607460 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2088e1-9936-459c-b273-f591d9169b2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:47.607737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2088e1-9936-459c-b273-f591d9169b2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:47.609734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bc2088e1-9936-459c-b273-f591d9169b2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:47.701514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77553457-8935-4635-8201-5be91c2d6f04 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:47.702981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-77553457-8935-4635-8201-5be91c2d6f04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:47.716133 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:47.740467 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77553457-8935-4635-8201-5be91c2d6f04 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:47.753724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:47.754388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:47.754720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:47.755001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] PATH_INFO: `/v3/projects/606ffec2caa54e2085b63485214ffdb6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:47.756180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=606ffec2caa54e2085b63485214ffdb6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:47.758172 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:47.774384 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2088e1-9936-459c-b273-f591d9169b2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:47.776400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2088e1-9936-459c-b273-f591d9169b2c service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:47.799877 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1093/2199] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:47 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:47.859890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '606ffec2caa54e2085b63485214ffdb6', 'request_id': 'req-58c829c3-939d-460b-a9fb-1e76bc07f858'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:47.877049 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '606ffec2caa54e2085b63485214ffdb6', 'request_id': 'req-58c829c3-939d-460b-a9fb-1e76bc07f858'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:47.895829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '606ffec2caa54e2085b63485214ffdb6', 'request_id': 'req-58c829c3-939d-460b-a9fb-1e76bc07f858'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:47.933959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] The token cache is being invalidate because project 606ffec2caa54e2085b63485214ffdb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:47.934852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:47.936996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:47.946369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1107/2200] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:47 2022] DELETE /identity/v3/projects/606ffec2caa54e2085b63485214ffdb6 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:47.948340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2088e1-9936-459c-b273-f591d9169b2c service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:47.970804 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2088e1-9936-459c-b273-f591d9169b2c service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:48.091800 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:48.092430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:48.092833 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:48.093113 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] PATH_INFO: `/v3/users/85c999b0c7184e4997138a20a636f072` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:48.094396 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=85c999b0c7184e4997138a20a636f072)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:48.096392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:48.126831 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:48.190611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85c999b0c7184e4997138a20a636f072'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:48.219405 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85c999b0c7184e4997138a20a636f072'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:48.248614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85c999b0c7184e4997138a20a636f072'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:48.279939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85c999b0c7184e4997138a20a636f072'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:48.301776 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85c999b0c7184e4997138a20a636f072'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:48.322122 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1094/2201] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:47 2022] DELETE /identity/v3/users/85c999b0c7184e4997138a20a636f072 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:16:48.853775 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:48.855422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:48.896817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c829c3-939d-460b-a9fb-1e76bc07f858 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:48.972209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:48.972849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:48.973128 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:48.973401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] PATH_INFO: `/v3/projects/e7f6b887fb204970a52afd3fad2e1e7f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:48.974527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7f6b887fb204970a52afd3fad2e1e7f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:48.976759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:49.031283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7f6b887fb204970a52afd3fad2e1e7f', 'request_id': 'req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:49.044767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7f6b887fb204970a52afd3fad2e1e7f', 'request_id': 'req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:49.054201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7f6b887fb204970a52afd3fad2e1e7f', 'request_id': 'req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:49.075023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] The token cache is being invalidate because project e7f6b887fb204970a52afd3fad2e1e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:49.075587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:49.076461 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:49.078192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1108/2202] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:48 2022] DELETE /identity/v3/projects/e7f6b887fb204970a52afd3fad2e1e7f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:49.082851 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:49.084488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:49.108947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54bed7-35d5-45b8-a0e4-7437f1fb86ea admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:49.187896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cca6b9-42bf-40a7-bf88-50fc87fa953a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:49.188630 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cca6b9-42bf-40a7-bf88-50fc87fa953a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:49.189003 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cca6b9-42bf-40a7-bf88-50fc87fa953a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:49.189338 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cca6b9-42bf-40a7-bf88-50fc87fa953a admin admin] PATH_INFO: `/v3/users/538c85e1067345a19908e94a43a9d095` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:49.190542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cca6b9-42bf-40a7-bf88-50fc87fa953a admin admin] RBAC: Authorizing `identity:delete_user(user_id=538c85e1067345a19908e94a43a9d095)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:49.192591 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cca6b9-42bf-40a7-bf88-50fc87fa953a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:49.255683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f7cca6b9-42bf-40a7-bf88-50fc87fa953a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '538c85e1067345a19908e94a43a9d095'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:49.271737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f7cca6b9-42bf-40a7-bf88-50fc87fa953a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '538c85e1067345a19908e94a43a9d095'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:49.292446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f7cca6b9-42bf-40a7-bf88-50fc87fa953a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '538c85e1067345a19908e94a43a9d095'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:49.295009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f7cca6b9-42bf-40a7-bf88-50fc87fa953a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '538c85e1067345a19908e94a43a9d095'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:49.308313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f7cca6b9-42bf-40a7-bf88-50fc87fa953a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '538c85e1067345a19908e94a43a9d095'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:49.316599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1095/2203] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:49 2022] DELETE /identity/v3/users/538c85e1067345a19908e94a43a9d095 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:49.987684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:49.989550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:50.005279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eec9dc-c26b-4e07-b345-bd42aea8d2fc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:50.023548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:50.108287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a24309-97f8-475b-b86f-b734ff57c648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:50.108939 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a24309-97f8-475b-b86f-b734ff57c648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:50.109223 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a24309-97f8-475b-b86f-b734ff57c648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:50.109500 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a24309-97f8-475b-b86f-b734ff57c648 admin admin] PATH_INFO: `/v3/projects/ed46e8d25d474a5ebec3007cf6ed6de4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:50.110635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a24309-97f8-475b-b86f-b734ff57c648 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed46e8d25d474a5ebec3007cf6ed6de4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:50.112916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a24309-97f8-475b-b86f-b734ff57c648 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:50.178915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b6a24309-97f8-475b-b86f-b734ff57c648 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed46e8d25d474a5ebec3007cf6ed6de4', 'request_id': 'req-b6a24309-97f8-475b-b86f-b734ff57c648'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:50.191781 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b6a24309-97f8-475b-b86f-b734ff57c648 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed46e8d25d474a5ebec3007cf6ed6de4', 'request_id': 'req-b6a24309-97f8-475b-b86f-b734ff57c648'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:50.201145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b6a24309-97f8-475b-b86f-b734ff57c648 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed46e8d25d474a5ebec3007cf6ed6de4', 'request_id': 'req-b6a24309-97f8-475b-b86f-b734ff57c648'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:50.216252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b6a24309-97f8-475b-b86f-b734ff57c648 admin admin] The token cache is being invalidate because project ed46e8d25d474a5ebec3007cf6ed6de4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:50.216554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b6a24309-97f8-475b-b86f-b734ff57c648 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:50.217764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b6a24309-97f8-475b-b86f-b734ff57c648 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:50.220327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1109/2204] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:49 2022] DELETE /identity/v3/projects/ed46e8d25d474a5ebec3007cf6ed6de4 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:50.225868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c89f4-d785-4e64-8395-7c3713aba1c1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:50.226348 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c89f4-d785-4e64-8395-7c3713aba1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:50.226458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c89f4-d785-4e64-8395-7c3713aba1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:50.269173 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-574c89f4-d785-4e64-8395-7c3713aba1c1 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:50.270733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-574c89f4-d785-4e64-8395-7c3713aba1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:50.287930 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:50.369275 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1096/2205] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:50.377705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385275e2-75e5-4af4-9df2-a66feb32b2ec None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:50.377987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385275e2-75e5-4af4-9df2-a66feb32b2ec None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:50.378261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385275e2-75e5-4af4-9df2-a66feb32b2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:50.400837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-385275e2-75e5-4af4-9df2-a66feb32b2ec None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:50.402642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-385275e2-75e5-4af4-9df2-a66feb32b2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:50.411255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1110/2206] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:50.415639 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c89f4-d785-4e64-8395-7c3713aba1c1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:50.432948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c89f4-d785-4e64-8395-7c3713aba1c1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:50.439433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b3d4e-f704-4411-8223-514e6b074cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:50.440057 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b3d4e-f704-4411-8223-514e6b074cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:50.440401 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b3d4e-f704-4411-8223-514e6b074cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:50.440674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b3d4e-f704-4411-8223-514e6b074cd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:50.441445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b3d4e-f704-4411-8223-514e6b074cd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:16:50.441654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b3d4e-f704-4411-8223-514e6b074cd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:50.443367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b3d4e-f704-4411-8223-514e6b074cd5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:50.464587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1097/2207] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:16:50.469369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385275e2-75e5-4af4-9df2-a66feb32b2ec None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:50.481638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385275e2-75e5-4af4-9df2-a66feb32b2ec None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:50.490308 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a5629-01b5-435b-b145-62cfd7c659ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:50.492415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a5629-01b5-435b-b145-62cfd7c659ca admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:50.492704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a5629-01b5-435b-b145-62cfd7c659ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:50.493043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a5629-01b5-435b-b145-62cfd7c659ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:50.493696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a5629-01b5-435b-b145-62cfd7c659ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:50.496974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a5629-01b5-435b-b145-62cfd7c659ca admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:50.563234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1111/2208] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:50 2022] POST /identity/v3/projects => generated 369 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:50.570420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b3d4e-f704-4411-8223-514e6b074cd5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:50.604361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b3d4e-f704-4411-8223-514e6b074cd5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:50.754622 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56ee15-5665-474e-8f44-c373cd33476d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:50.763857 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56ee15-5665-474e-8f44-c373cd33476d admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:50.765174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56ee15-5665-474e-8f44-c373cd33476d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:50.765793 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56ee15-5665-474e-8f44-c373cd33476d admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:50.768161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56ee15-5665-474e-8f44-c373cd33476d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:50.774224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56ee15-5665-474e-8f44-c373cd33476d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:50.813344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1098/2209] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:50 2022] POST /identity/v3/users => generated 285 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:50.817428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a5629-01b5-435b-b145-62cfd7c659ca admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:50.828870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a5629-01b5-435b-b145-62cfd7c659ca admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:50.841274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235c8d0-9425-4e37-b336-8093c123cd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:50.841274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235c8d0-9425-4e37-b336-8093c123cd08 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:50.841274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235c8d0-9425-4e37-b336-8093c123cd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:50.841274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235c8d0-9425-4e37-b336-8093c123cd08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:50.841720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235c8d0-9425-4e37-b336-8093c123cd08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:50.841720 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235c8d0-9425-4e37-b336-8093c123cd08 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:50.851019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1112/2210] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:50.855383 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56ee15-5665-474e-8f44-c373cd33476d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:50.869885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56ee15-5665-474e-8f44-c373cd33476d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:50.880380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a03f4b-1426-43da-a016-0078a5b85af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:50.881441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a03f4b-1426-43da-a016-0078a5b85af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:50.881790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a03f4b-1426-43da-a016-0078a5b85af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:50.883174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a03f4b-1426-43da-a016-0078a5b85af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:50.884329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a03f4b-1426-43da-a016-0078a5b85af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:50.887932 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a03f4b-1426-43da-a016-0078a5b85af5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:50.912504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1099/2211] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:50.914636 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235c8d0-9425-4e37-b336-8093c123cd08 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:50.928157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235c8d0-9425-4e37-b336-8093c123cd08 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:50.943621 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab629d-f776-43d6-8d19-9acb7e401f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:50.944301 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab629d-f776-43d6-8d19-9acb7e401f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:50.944510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab629d-f776-43d6-8d19-9acb7e401f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:50.944790 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab629d-f776-43d6-8d19-9acb7e401f1a admin admin] PATH_INFO: `/v3/projects/e6f3b1b36a4b441b914525415c19e9ba/users/9824d4e028f940a4bdd15f9f56aa8299/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:50.978465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab629d-f776-43d6-8d19-9acb7e401f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6f3b1b36a4b441b914525415c19e9ba, user_id=9824d4e028f940a4bdd15f9f56aa8299, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:50.985252 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab629d-f776-43d6-8d19-9acb7e401f1a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:51.019576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1113/2212] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:50 2022] PUT /identity/v3/projects/e6f3b1b36a4b441b914525415c19e9ba/users/9824d4e028f940a4bdd15f9f56aa8299/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:51.110972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301fda4-273d-431d-9220-922be83c9ab9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:51.111714 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301fda4-273d-431d-9220-922be83c9ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:51.112128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301fda4-273d-431d-9220-922be83c9ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:51.181410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a301fda4-273d-431d-9220-922be83c9ab9 None None] MFA Rules not processed for user `9824d4e028f940a4bdd15f9f56aa8299`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:51.182698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a301fda4-273d-431d-9220-922be83c9ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:51.419348 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1100/2213] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:52.447914 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab629d-f776-43d6-8d19-9acb7e401f1a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:52.450024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-eaab629d-f776-43d6-8d19-9acb7e401f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:52.537774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:52.580866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab629d-f776-43d6-8d19-9acb7e401f1a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:52.592300 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:52.594184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:52.594860 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:52.595267 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] PATH_INFO: `/v3/users/c03bbf274f3f477b9d5308d44b49ca54` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:52.601758 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03bbf274f3f477b9d5308d44b49ca54)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:52.604742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:52.709346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03bbf274f3f477b9d5308d44b49ca54'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:52.722004 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03bbf274f3f477b9d5308d44b49ca54'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:52.722996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03bbf274f3f477b9d5308d44b49ca54'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:52.743678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03bbf274f3f477b9d5308d44b49ca54'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:52.758093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03bbf274f3f477b9d5308d44b49ca54'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:52.770263 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1114/2214] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:52 2022] DELETE /identity/v3/users/c03bbf274f3f477b9d5308d44b49ca54 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:52.775503 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d195ca-73c7-48de-b674-30f787ab3360 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:52.776558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d195ca-73c7-48de-b674-30f787ab3360 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:52.777099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d195ca-73c7-48de-b674-30f787ab3360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:52.813050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a4d195ca-73c7-48de-b674-30f787ab3360 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:52.814832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a4d195ca-73c7-48de-b674-30f787ab3360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:52.825595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:52.901781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1101/2215] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:52.912071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:52.913485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:52.976585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230811e1-116f-4641-a93a-1feafc4cdfcb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:52.981119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab911c65-5b11-49c1-a98b-3e9e89cfe954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:52.981958 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab911c65-5b11-49c1-a98b-3e9e89cfe954 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:52.982241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab911c65-5b11-49c1-a98b-3e9e89cfe954 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:52.982516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab911c65-5b11-49c1-a98b-3e9e89cfe954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:53.005999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab911c65-5b11-49c1-a98b-3e9e89cfe954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:53.011672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab911c65-5b11-49c1-a98b-3e9e89cfe954 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:53.031397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1115/2216] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:16:53.336982 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d195ca-73c7-48de-b674-30f787ab3360 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:53.338771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a4d195ca-73c7-48de-b674-30f787ab3360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:53.353708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d195ca-73c7-48de-b674-30f787ab3360 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:53.361870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:53.363005 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:53.363541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:53.364042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] PATH_INFO: `/v3/projects/3586ecca9d18472491fe46f6da0dcf0a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:53.366020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3586ecca9d18472491fe46f6da0dcf0a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:53.368419 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:53.439186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3586ecca9d18472491fe46f6da0dcf0a', 'request_id': 'req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:53.447264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3586ecca9d18472491fe46f6da0dcf0a', 'request_id': 'req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:53.454180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3586ecca9d18472491fe46f6da0dcf0a', 'request_id': 'req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:53.473131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] The token cache is being invalidate because project 3586ecca9d18472491fe46f6da0dcf0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:53.473517 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:53.474226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:53.478759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1102/2217] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:53 2022] DELETE /identity/v3/projects/3586ecca9d18472491fe46f6da0dcf0a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:53.480117 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab911c65-5b11-49c1-a98b-3e9e89cfe954 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:53.492095 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab911c65-5b11-49c1-a98b-3e9e89cfe954 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:53.572979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:53.573621 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:53.573896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:53.574166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] PATH_INFO: `/v3/users/af8c7a5a97a94fa9b2dfdabfde169039` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:53.588039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] RBAC: Authorizing `identity:delete_user(user_id=af8c7a5a97a94fa9b2dfdabfde169039)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:53.590664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:53.603538 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:53.697923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af8c7a5a97a94fa9b2dfdabfde169039'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:53.712622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af8c7a5a97a94fa9b2dfdabfde169039'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:53.713420 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af8c7a5a97a94fa9b2dfdabfde169039'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:53.733944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af8c7a5a97a94fa9b2dfdabfde169039'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:53.742840 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af8c7a5a97a94fa9b2dfdabfde169039'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:53.751939 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1116/2218] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:53 2022] DELETE /identity/v3/users/af8c7a5a97a94fa9b2dfdabfde169039 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:54.489899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:54.491789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:54.515136 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2486ed5-c9af-4ff5-b91b-f610a6603c7b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:54.665608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:54.666511 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:54.666847 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:54.667224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] PATH_INFO: `/v3/projects/6313571b417a4798908db239cab1fa7e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:54.669573 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] RBAC: Authorizing `identity:delete_project(project_id=6313571b417a4798908db239cab1fa7e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:54.672658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:54.743912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:54.771278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6313571b417a4798908db239cab1fa7e', 'request_id': 'req-0137cb62-94f7-4594-95e8-1523c34cf7da'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:54.784796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6313571b417a4798908db239cab1fa7e', 'request_id': 'req-0137cb62-94f7-4594-95e8-1523c34cf7da'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:54.792721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6313571b417a4798908db239cab1fa7e', 'request_id': 'req-0137cb62-94f7-4594-95e8-1523c34cf7da'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:54.810702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] The token cache is being invalidate because project 6313571b417a4798908db239cab1fa7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:54.811361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:54.812215 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:54.814213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1103/2219] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:54 2022] DELETE /identity/v3/projects/6313571b417a4798908db239cab1fa7e => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:54.817948 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:54.820754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:54.846662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6666d40-3d2d-4d03-928c-d8fb673eb10d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:54.976354 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabeb2c-85c0-427a-aae0-8d37c1879c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:54.977188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabeb2c-85c0-427a-aae0-8d37c1879c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:54.977565 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabeb2c-85c0-427a-aae0-8d37c1879c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:54.977957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabeb2c-85c0-427a-aae0-8d37c1879c9c admin admin] PATH_INFO: `/v3/users/4a2fa9a5b39e4e2d9978eeb9a064877a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:54.981546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabeb2c-85c0-427a-aae0-8d37c1879c9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a2fa9a5b39e4e2d9978eeb9a064877a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:54.984296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabeb2c-85c0-427a-aae0-8d37c1879c9c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:55.097503 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2eabeb2c-85c0-427a-aae0-8d37c1879c9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a2fa9a5b39e4e2d9978eeb9a064877a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:55.110538 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2eabeb2c-85c0-427a-aae0-8d37c1879c9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a2fa9a5b39e4e2d9978eeb9a064877a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:55.112226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2eabeb2c-85c0-427a-aae0-8d37c1879c9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a2fa9a5b39e4e2d9978eeb9a064877a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:55.134032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2eabeb2c-85c0-427a-aae0-8d37c1879c9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a2fa9a5b39e4e2d9978eeb9a064877a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:55.142270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2eabeb2c-85c0-427a-aae0-8d37c1879c9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a2fa9a5b39e4e2d9978eeb9a064877a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:55.158609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1117/2220] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:16:54 2022] DELETE /identity/v3/users/4a2fa9a5b39e4e2d9978eeb9a064877a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:16:55.674626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:55.677000 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:55.750378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137cb62-94f7-4594-95e8-1523c34cf7da admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:55.873185 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc95643-a469-415c-a767-33b4012ff06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:55.875947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95643-a469-415c-a767-33b4012ff06f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:55.875947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95643-a469-415c-a767-33b4012ff06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:55.875947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95643-a469-415c-a767-33b4012ff06f admin admin] PATH_INFO: `/v3/projects/8dfb298e3252405bae02d6921705db20` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:55.875947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc95643-a469-415c-a767-33b4012ff06f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dfb298e3252405bae02d6921705db20)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:55.879616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc95643-a469-415c-a767-33b4012ff06f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:55.981449 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2dc95643-a469-415c-a767-33b4012ff06f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dfb298e3252405bae02d6921705db20', 'request_id': 'req-2dc95643-a469-415c-a767-33b4012ff06f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:55.992522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2dc95643-a469-415c-a767-33b4012ff06f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dfb298e3252405bae02d6921705db20', 'request_id': 'req-2dc95643-a469-415c-a767-33b4012ff06f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:56.004637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2dc95643-a469-415c-a767-33b4012ff06f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dfb298e3252405bae02d6921705db20', 'request_id': 'req-2dc95643-a469-415c-a767-33b4012ff06f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:56.029071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2dc95643-a469-415c-a767-33b4012ff06f admin admin] The token cache is being invalidate because project 8dfb298e3252405bae02d6921705db20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:16:56.029919 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2dc95643-a469-415c-a767-33b4012ff06f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:56.031010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2dc95643-a469-415c-a767-33b4012ff06f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:16:56.034693 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1104/2221] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:16:55 2022] DELETE /identity/v3/projects/8dfb298e3252405bae02d6921705db20 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:56.039821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092db6e1-c54e-44d0-b631-5320eaf434eb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:56.040388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092db6e1-c54e-44d0-b631-5320eaf434eb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:56.040519 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092db6e1-c54e-44d0-b631-5320eaf434eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:56.061969 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:56.083862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-092db6e1-c54e-44d0-b631-5320eaf434eb None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:56.085988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-092db6e1-c54e-44d0-b631-5320eaf434eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:56.232704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1118/2222] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:56.241737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156478b-8a9d-42f4-9db1-84cab1155991 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:56.242021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156478b-8a9d-42f4-9db1-84cab1155991 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:56.242306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156478b-8a9d-42f4-9db1-84cab1155991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:56.304100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8156478b-8a9d-42f4-9db1-84cab1155991 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:56.308283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8156478b-8a9d-42f4-9db1-84cab1155991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:56.345410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1105/2223] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:16:56.349640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092db6e1-c54e-44d0-b631-5320eaf434eb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:56.362318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092db6e1-c54e-44d0-b631-5320eaf434eb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:56.369895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4e3c4-303c-4c3e-9773-08464ea0ae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:56.371439 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4e3c4-303c-4c3e-9773-08464ea0ae62 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:56.372279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4e3c4-303c-4c3e-9773-08464ea0ae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:56.372889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4e3c4-303c-4c3e-9773-08464ea0ae62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:56.374165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4e3c4-303c-4c3e-9773-08464ea0ae62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:16:56.375130 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4e3c4-303c-4c3e-9773-08464ea0ae62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:56.377650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4e3c4-303c-4c3e-9773-08464ea0ae62 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:56.400402 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1119/2224] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:16:56.405198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156478b-8a9d-42f4-9db1-84cab1155991 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:56.417825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156478b-8a9d-42f4-9db1-84cab1155991 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:56.431783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b037d2-f622-409f-b394-f49aa0f68afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:56.432459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b037d2-f622-409f-b394-f49aa0f68afa admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:56.432743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b037d2-f622-409f-b394-f49aa0f68afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:56.433030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b037d2-f622-409f-b394-f49aa0f68afa admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:56.433722 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b037d2-f622-409f-b394-f49aa0f68afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:56.435761 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b037d2-f622-409f-b394-f49aa0f68afa admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:56.495569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1106/2225] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:16:56 2022] POST /identity/v3/projects => generated 369 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:16:56.501190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4e3c4-303c-4c3e-9773-08464ea0ae62 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:56.530364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4e3c4-303c-4c3e-9773-08464ea0ae62 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:56.802296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51278272-8f51-49b3-a5f8-1902e8dac191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:56.803981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51278272-8f51-49b3-a5f8-1902e8dac191 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:56.805010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51278272-8f51-49b3-a5f8-1902e8dac191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:56.805644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51278272-8f51-49b3-a5f8-1902e8dac191 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:56.806432 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51278272-8f51-49b3-a5f8-1902e8dac191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:56.808581 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51278272-8f51-49b3-a5f8-1902e8dac191 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:56.854162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1120/2226] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:16:56 2022] POST /identity/v3/users => generated 285 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:16:56.854847 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b037d2-f622-409f-b394-f49aa0f68afa admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:56.870089 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b037d2-f622-409f-b394-f49aa0f68afa admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:56.882779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f7043-9d5c-4894-a3a4-e50a7970732d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:56.883523 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f7043-9d5c-4894-a3a4-e50a7970732d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:56.883799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f7043-9d5c-4894-a3a4-e50a7970732d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:56.884070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f7043-9d5c-4894-a3a4-e50a7970732d admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:56.885327 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f7043-9d5c-4894-a3a4-e50a7970732d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:56.887735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f7043-9d5c-4894-a3a4-e50a7970732d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:56.910050 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1107/2227] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:16:56.911964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51278272-8f51-49b3-a5f8-1902e8dac191 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:56.925922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51278272-8f51-49b3-a5f8-1902e8dac191 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:56.940807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816d76f-059e-4ab1-96be-f5e0a9e2e9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:56.940807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816d76f-059e-4ab1-96be-f5e0a9e2e9c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:56.940807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816d76f-059e-4ab1-96be-f5e0a9e2e9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:56.940807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816d76f-059e-4ab1-96be-f5e0a9e2e9c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:56.941339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816d76f-059e-4ab1-96be-f5e0a9e2e9c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:56.941339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816d76f-059e-4ab1-96be-f5e0a9e2e9c9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:56.975959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1121/2228] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:16:56.979812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f7043-9d5c-4894-a3a4-e50a7970732d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:16:56.999743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f7043-9d5c-4894-a3a4-e50a7970732d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:16:57.013867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc3382-d54e-42e3-9663-cbeb7a8e4253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:16:57.014573 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc3382-d54e-42e3-9663-cbeb7a8e4253 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:57.014898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc3382-d54e-42e3-9663-cbeb7a8e4253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:57.015154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc3382-d54e-42e3-9663-cbeb7a8e4253 admin admin] PATH_INFO: `/v3/projects/b4a91ae8f1b54a89b153b4c6030ff455/users/0dd9f9a08c02453e8a5a8e40ebf72b3a/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:57.035940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:57.041879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc3382-d54e-42e3-9663-cbeb7a8e4253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a91ae8f1b54a89b153b4c6030ff455, user_id=0dd9f9a08c02453e8a5a8e40ebf72b3a, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:16:57.053511 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc3382-d54e-42e3-9663-cbeb7a8e4253 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:16:57.113805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1108/2229] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:16:56 2022] PUT /identity/v3/projects/b4a91ae8f1b54a89b153b4c6030ff455/users/0dd9f9a08c02453e8a5a8e40ebf72b3a/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:16:57.117659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd4a15-8b88-44d9-a985-abc4b433f1a5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:16:57.118789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd4a15-8b88-44d9-a985-abc4b433f1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:16:57.119206 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd4a15-8b88-44d9-a985-abc4b433f1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:16:57.176867 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-58bd4a15-8b88-44d9-a985-abc4b433f1a5 None None] MFA Rules not processed for user `0dd9f9a08c02453e8a5a8e40ebf72b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:16:57.180025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-58bd4a15-8b88-44d9-a985-abc4b433f1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:16:57.385744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:16:57.420233 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1122/2230] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:16:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:03.717200 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d644da4-8684-41b0-8a65-2b9381920026 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:03.717807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d644da4-8684-41b0-8a65-2b9381920026 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:03.717807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d644da4-8684-41b0-8a65-2b9381920026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:03.745962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8d644da4-8684-41b0-8a65-2b9381920026 None None] MFA Rules not processed for user `dcb88983a8e1439588644b71170d7362`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:03.749779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8d644da4-8684-41b0-8a65-2b9381920026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:03.871427 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1109/2231] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:05.213567 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae6616-fb82-4ce8-9cb0-0f5727fc4fff None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:05.214190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae6616-fb82-4ce8-9cb0-0f5727fc4fff None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:05.214470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae6616-fb82-4ce8-9cb0-0f5727fc4fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:05.235925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7eae6616-fb82-4ce8-9cb0-0f5727fc4fff None None] MFA Rules not processed for user `21700a3c2ff24fc6a3bb5db8bf283f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:05.237171 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7eae6616-fb82-4ce8-9cb0-0f5727fc4fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:05.314558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1123/2232] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:08.913101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d644da4-8684-41b0-8a65-2b9381920026 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:08.915406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8d644da4-8684-41b0-8a65-2b9381920026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:09.015756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d644da4-8684-41b0-8a65-2b9381920026 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:09.019575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d2c63-4a1d-40be-aeb4-38273a696d34 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ef6cd64182474787134e3227beb3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:09.020212 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d2c63-4a1d-40be-aeb4-38273a696d34 service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:09.020500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d2c63-4a1d-40be-aeb4-38273a696d34 service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:09.020792 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d2c63-4a1d-40be-aeb4-38273a696d34 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:09.022592 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-312d2c63-4a1d-40be-aeb4-38273a696d34 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:09.108176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d2c63-4a1d-40be-aeb4-38273a696d34 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:09.109775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d2c63-4a1d-40be-aeb4-38273a696d34 service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:09.128791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1110/2233] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:17:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:17:09.170262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae6616-fb82-4ce8-9cb0-0f5727fc4fff None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:09.180472 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae6616-fb82-4ce8-9cb0-0f5727fc4fff None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:09.185612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc580cae-4f75-4dff-b847-2859ba053cb1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ef6cd64182474787134e3227beb3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:09.186226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc580cae-4f75-4dff-b847-2859ba053cb1 service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:09.186502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc580cae-4f75-4dff-b847-2859ba053cb1 service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:09.186790 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc580cae-4f75-4dff-b847-2859ba053cb1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:09.196216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc580cae-4f75-4dff-b847-2859ba053cb1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:09.197684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc580cae-4f75-4dff-b847-2859ba053cb1 service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:09.207751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1124/2234] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:17:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:17:19.115371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:17:19.203313 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=32 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:17:21.063485 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d2c63-4a1d-40be-aeb4-38273a696d34 service glance] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:21.065175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-312d2c63-4a1d-40be-aeb4-38273a696d34 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:21.137099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d2c63-4a1d-40be-aeb4-38273a696d34 service glance] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:21.141857 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:21.142513 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:21.142798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:21.143087 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] PATH_INFO: `/v3/users/5a0abb18539c4116b48fe697dd3aa82d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:21.144236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a0abb18539c4116b48fe697dd3aa82d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:21.146184 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:21.193258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a0abb18539c4116b48fe697dd3aa82d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:21.203588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a0abb18539c4116b48fe697dd3aa82d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:21.216823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a0abb18539c4116b48fe697dd3aa82d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:21.217652 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a0abb18539c4116b48fe697dd3aa82d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:21.225615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a0abb18539c4116b48fe697dd3aa82d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:21.231708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1111/2235] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:17:21 2022] DELETE /identity/v3/users/5a0abb18539c4116b48fe697dd3aa82d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:17:21.236236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1886cd4-1531-407f-9002-250ead8fe4e6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:21.236721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1886cd4-1531-407f-9002-250ead8fe4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:21.236858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1886cd4-1531-407f-9002-250ead8fe4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:21.257048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f1886cd4-1531-407f-9002-250ead8fe4e6 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:21.258271 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f1886cd4-1531-407f-9002-250ead8fe4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:21.325019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1125/2236] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:21.334895 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:21.336451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:21.402923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25828fe-7b3d-4ba8-84e2-33ca6fa3bfc9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:21.406867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41b60f-1d95-426a-a7de-00c0a4f26652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:21.407568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41b60f-1d95-426a-a7de-00c0a4f26652 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:21.407946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41b60f-1d95-426a-a7de-00c0a4f26652 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:21.408335 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41b60f-1d95-426a-a7de-00c0a4f26652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:21.419808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41b60f-1d95-426a-a7de-00c0a4f26652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:21.421633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41b60f-1d95-426a-a7de-00c0a4f26652 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:21.431119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1112/2237] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:17:21.741975 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1886cd4-1531-407f-9002-250ead8fe4e6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:21.743814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f1886cd4-1531-407f-9002-250ead8fe4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:21.766525 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1886cd4-1531-407f-9002-250ead8fe4e6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:21.771622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f816e-eef7-43b8-abc8-000990d7a278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:21.772299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f816e-eef7-43b8-abc8-000990d7a278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:21.772578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f816e-eef7-43b8-abc8-000990d7a278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:21.772855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f816e-eef7-43b8-abc8-000990d7a278 admin admin] PATH_INFO: `/v3/projects/b59568c5604c4bfb9564fff58a162f6d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:21.773988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f816e-eef7-43b8-abc8-000990d7a278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b59568c5604c4bfb9564fff58a162f6d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:21.776035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f816e-eef7-43b8-abc8-000990d7a278 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:21.830784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8b9f816e-eef7-43b8-abc8-000990d7a278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b59568c5604c4bfb9564fff58a162f6d', 'request_id': 'req-8b9f816e-eef7-43b8-abc8-000990d7a278'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:21.841185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8b9f816e-eef7-43b8-abc8-000990d7a278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b59568c5604c4bfb9564fff58a162f6d', 'request_id': 'req-8b9f816e-eef7-43b8-abc8-000990d7a278'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:21.849950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8b9f816e-eef7-43b8-abc8-000990d7a278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b59568c5604c4bfb9564fff58a162f6d', 'request_id': 'req-8b9f816e-eef7-43b8-abc8-000990d7a278'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:21.865616 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8b9f816e-eef7-43b8-abc8-000990d7a278 admin admin] The token cache is being invalidate because project b59568c5604c4bfb9564fff58a162f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:17:21.866161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8b9f816e-eef7-43b8-abc8-000990d7a278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:21.866843 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8b9f816e-eef7-43b8-abc8-000990d7a278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:21.868734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1126/2238] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:17:21 2022] DELETE /identity/v3/projects/b59568c5604c4bfb9564fff58a162f6d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:17:21.874284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007adc6-fb9d-4353-9845-6d8d197f0338 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:21.874573 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007adc6-fb9d-4353-9845-6d8d197f0338 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:21.874852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007adc6-fb9d-4353-9845-6d8d197f0338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:21.894560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b007adc6-fb9d-4353-9845-6d8d197f0338 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:21.895812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b007adc6-fb9d-4353-9845-6d8d197f0338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:21.916861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:17:21.967718 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1113/2239] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:21.977486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04452d-57d3-4a58-9907-60fc8b64aecf None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:21.977749 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04452d-57d3-4a58-9907-60fc8b64aecf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:21.978018 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04452d-57d3-4a58-9907-60fc8b64aecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:22.004021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6d04452d-57d3-4a58-9907-60fc8b64aecf None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:22.005250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6d04452d-57d3-4a58-9907-60fc8b64aecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:22.018614 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1127/2240] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:22.025423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007adc6-fb9d-4353-9845-6d8d197f0338 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:22.035139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007adc6-fb9d-4353-9845-6d8d197f0338 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:22.040005 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867343f-5c8f-4d43-b6e7-4a7c3e70d3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:22.040697 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867343f-5c8f-4d43-b6e7-4a7c3e70d3cf admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:22.041246 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867343f-5c8f-4d43-b6e7-4a7c3e70d3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:22.041655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867343f-5c8f-4d43-b6e7-4a7c3e70d3cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:22.042443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867343f-5c8f-4d43-b6e7-4a7c3e70d3cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:17:22.042806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867343f-5c8f-4d43-b6e7-4a7c3e70d3cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:22.044978 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867343f-5c8f-4d43-b6e7-4a7c3e70d3cf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:22.059773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1114/2241] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:17:22.069107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04452d-57d3-4a58-9907-60fc8b64aecf None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:22.076089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04452d-57d3-4a58-9907-60fc8b64aecf None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:22.081146 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bd40f-d2e7-4752-b04c-3045c7c4c6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:22.081683 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bd40f-d2e7-4752-b04c-3045c7c4c6f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:22.081958 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bd40f-d2e7-4752-b04c-3045c7c4c6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:22.082234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bd40f-d2e7-4752-b04c-3045c7c4c6f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:22.082853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bd40f-d2e7-4752-b04c-3045c7c4c6f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:22.084773 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bd40f-d2e7-4752-b04c-3045c7c4c6f5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:22.115191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1128/2242] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:17:22 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:17:22.119567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867343f-5c8f-4d43-b6e7-4a7c3e70d3cf admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:22.129599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867343f-5c8f-4d43-b6e7-4a7c3e70d3cf admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:22.187930 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e73e0-ce66-4d73-8ea8-62ef7e202071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:22.188548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e73e0-ce66-4d73-8ea8-62ef7e202071 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:22.188829 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e73e0-ce66-4d73-8ea8-62ef7e202071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:22.189103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e73e0-ce66-4d73-8ea8-62ef7e202071 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:22.189745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e73e0-ce66-4d73-8ea8-62ef7e202071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:22.191716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e73e0-ce66-4d73-8ea8-62ef7e202071 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:22.212349 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1115/2243] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:17:22 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:17:22.217599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bd40f-d2e7-4752-b04c-3045c7c4c6f5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:22.228238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bd40f-d2e7-4752-b04c-3045c7c4c6f5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:22.233150 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9ecd9-a578-4f4b-b69b-cf61d471a3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:22.233760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9ecd9-a578-4f4b-b69b-cf61d471a3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:22.234043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9ecd9-a578-4f4b-b69b-cf61d471a3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:22.234316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9ecd9-a578-4f4b-b69b-cf61d471a3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:22.234987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c9ecd9-a578-4f4b-b69b-cf61d471a3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:22.236643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c9ecd9-a578-4f4b-b69b-cf61d471a3d9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:22.247322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1129/2244] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:17:22.251445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e73e0-ce66-4d73-8ea8-62ef7e202071 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:22.262385 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e73e0-ce66-4d73-8ea8-62ef7e202071 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:22.267587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81aafc1-70c3-499e-ac3a-cfe7c0b3bf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:22.268206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81aafc1-70c3-499e-ac3a-cfe7c0b3bf5f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:22.268486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81aafc1-70c3-499e-ac3a-cfe7c0b3bf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:22.268759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81aafc1-70c3-499e-ac3a-cfe7c0b3bf5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:22.269475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81aafc1-70c3-499e-ac3a-cfe7c0b3bf5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:22.271144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81aafc1-70c3-499e-ac3a-cfe7c0b3bf5f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:22.288113 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1116/2245] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:17:22.290376 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9ecd9-a578-4f4b-b69b-cf61d471a3d9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:22.301578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9ecd9-a578-4f4b-b69b-cf61d471a3d9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:22.306455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5471624-b82f-45cf-8d11-ff6f96c94aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:22.307163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5471624-b82f-45cf-8d11-ff6f96c94aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:22.307473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5471624-b82f-45cf-8d11-ff6f96c94aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:22.307750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5471624-b82f-45cf-8d11-ff6f96c94aa0 admin admin] PATH_INFO: `/v3/projects/72081ed54ba24a16a2e078c333c8ba07/users/9eb1f32b0586462f82ffd2f5e8abbd11/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:22.322847 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5471624-b82f-45cf-8d11-ff6f96c94aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72081ed54ba24a16a2e078c333c8ba07, user_id=9eb1f32b0586462f82ffd2f5e8abbd11, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:22.326408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5471624-b82f-45cf-8d11-ff6f96c94aa0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:22.339818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1130/2246] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:17:22 2022] PUT /identity/v3/projects/72081ed54ba24a16a2e078c333c8ba07/users/9eb1f32b0586462f82ffd2f5e8abbd11/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:17:22.350209 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6550cf2-5cb8-47a3-abc1-65caefa6fa41 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:22.350503 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6550cf2-5cb8-47a3-abc1-65caefa6fa41 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:22.350867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6550cf2-5cb8-47a3-abc1-65caefa6fa41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:22.382248 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d6550cf2-5cb8-47a3-abc1-65caefa6fa41 None None] MFA Rules not processed for user `9eb1f32b0586462f82ffd2f5e8abbd11`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:22.383828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d6550cf2-5cb8-47a3-abc1-65caefa6fa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:22.451885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1117/2247] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:22.462578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5471624-b82f-45cf-8d11-ff6f96c94aa0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:22.464432 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c5471624-b82f-45cf-8d11-ff6f96c94aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:22.487318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:17:22.526213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5471624-b82f-45cf-8d11-ff6f96c94aa0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:22.530021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2f33b-0c9f-45ca-b538-8a86df1d0879 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:22.530628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2f33b-0c9f-45ca-b538-8a86df1d0879 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:22.530905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2f33b-0c9f-45ca-b538-8a86df1d0879 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:22.531212 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2f33b-0c9f-45ca-b538-8a86df1d0879 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:22.539557 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2f33b-0c9f-45ca-b538-8a86df1d0879 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:22.541021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2f33b-0c9f-45ca-b538-8a86df1d0879 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:22.550908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1131/2248] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:17:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:17:29.083116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1887d3-6a65-4102-9d6d-e77859579fa7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:29.083603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1887d3-6a65-4102-9d6d-e77859579fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:29.083822 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1887d3-6a65-4102-9d6d-e77859579fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:29.103367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ac1887d3-6a65-4102-9d6d-e77859579fa7 None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:29.104687 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ac1887d3-6a65-4102-9d6d-e77859579fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:29.118695 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:17:29.165647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1118/2249] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 9 13:17:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:29.179578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2f33b-0c9f-45ca-b538-8a86df1d0879 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:29.180993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3cf2f33b-0c9f-45ca-b538-8a86df1d0879 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:29.252329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2f33b-0c9f-45ca-b538-8a86df1d0879 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:29.257408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73771701-de4c-4e0b-a859-6283bc4a670c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:29.258071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73771701-de4c-4e0b-a859-6283bc4a670c service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:29.258356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73771701-de4c-4e0b-a859-6283bc4a670c service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:29.258630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73771701-de4c-4e0b-a859-6283bc4a670c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:29.267847 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73771701-de4c-4e0b-a859-6283bc4a670c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:29.269411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73771701-de4c-4e0b-a859-6283bc4a670c service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:29.278295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1132/2250] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:17:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:17:34.700456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1887d3-6a65-4102-9d6d-e77859579fa7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:34.702569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ac1887d3-6a65-4102-9d6d-e77859579fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:34.797016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1887d3-6a65-4102-9d6d-e77859579fa7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:34.801528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844e82b-ca12-42cc-924c-d881a54726e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:34.802324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844e82b-ca12-42cc-924c-d881a54726e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:34.802609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844e82b-ca12-42cc-924c-d881a54726e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:34.802888 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844e82b-ca12-42cc-924c-d881a54726e2 admin admin] PATH_INFO: `/v3/users/d73fdae39484489facaf6831430cd4d7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:34.804168 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844e82b-ca12-42cc-924c-d881a54726e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d73fdae39484489facaf6831430cd4d7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:34.806368 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844e82b-ca12-42cc-924c-d881a54726e2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:34.876296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d844e82b-ca12-42cc-924c-d881a54726e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd73fdae39484489facaf6831430cd4d7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:34.888989 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d844e82b-ca12-42cc-924c-d881a54726e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd73fdae39484489facaf6831430cd4d7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:34.907664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d844e82b-ca12-42cc-924c-d881a54726e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd73fdae39484489facaf6831430cd4d7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:34.908590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d844e82b-ca12-42cc-924c-d881a54726e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd73fdae39484489facaf6831430cd4d7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:34.916584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d844e82b-ca12-42cc-924c-d881a54726e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd73fdae39484489facaf6831430cd4d7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:34.926848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1119/2251] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:17:34 2022] DELETE /identity/v3/users/d73fdae39484489facaf6831430cd4d7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:17:35.110030 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73771701-de4c-4e0b-a859-6283bc4a670c service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:35.124597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73771701-de4c-4e0b-a859-6283bc4a670c service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:35.186580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af26b7-38ef-4f7f-869e-190e69cb6f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:35.187573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af26b7-38ef-4f7f-869e-190e69cb6f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:35.187883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af26b7-38ef-4f7f-869e-190e69cb6f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:35.188159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af26b7-38ef-4f7f-869e-190e69cb6f80 admin admin] PATH_INFO: `/v3/projects/ca38013a20eb4c8e830e391cc06f485f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:35.189306 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af26b7-38ef-4f7f-869e-190e69cb6f80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca38013a20eb4c8e830e391cc06f485f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:35.191329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af26b7-38ef-4f7f-869e-190e69cb6f80 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:35.238628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c5af26b7-38ef-4f7f-869e-190e69cb6f80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca38013a20eb4c8e830e391cc06f485f', 'request_id': 'req-c5af26b7-38ef-4f7f-869e-190e69cb6f80'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:35.242629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:17:35.248068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c5af26b7-38ef-4f7f-869e-190e69cb6f80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca38013a20eb4c8e830e391cc06f485f', 'request_id': 'req-c5af26b7-38ef-4f7f-869e-190e69cb6f80'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:35.255571 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c5af26b7-38ef-4f7f-869e-190e69cb6f80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca38013a20eb4c8e830e391cc06f485f', 'request_id': 'req-c5af26b7-38ef-4f7f-869e-190e69cb6f80'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:35.270773 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c5af26b7-38ef-4f7f-869e-190e69cb6f80 admin admin] The token cache is being invalidate because project ca38013a20eb4c8e830e391cc06f485f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:17:35.271074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c5af26b7-38ef-4f7f-869e-190e69cb6f80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:35.271812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c5af26b7-38ef-4f7f-869e-190e69cb6f80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:35.273312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1133/2252] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:17:35 2022] DELETE /identity/v3/projects/ca38013a20eb4c8e830e391cc06f485f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:17:35.279888 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af626530-0356-4944-9478-7a42309f9d01 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:35.280206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af626530-0356-4944-9478-7a42309f9d01 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:35.280486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af626530-0356-4944-9478-7a42309f9d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:35.310093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-af626530-0356-4944-9478-7a42309f9d01 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:35.312081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-af626530-0356-4944-9478-7a42309f9d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:35.363874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:17:35.388138 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1120/2253] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:35.396920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad159c58-b387-46b7-8ad9-b21904e5ed04 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:35.397338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad159c58-b387-46b7-8ad9-b21904e5ed04 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:35.397626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad159c58-b387-46b7-8ad9-b21904e5ed04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:35.418689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ad159c58-b387-46b7-8ad9-b21904e5ed04 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:35.420043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ad159c58-b387-46b7-8ad9-b21904e5ed04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:35.428394 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1134/2254] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:35.432580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af626530-0356-4944-9478-7a42309f9d01 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:35.442767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af626530-0356-4944-9478-7a42309f9d01 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:35.447401 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1a8ae-6e1a-487d-b7c8-87ff3cf05945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:35.448028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1a8ae-6e1a-487d-b7c8-87ff3cf05945 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:35.448302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1a8ae-6e1a-487d-b7c8-87ff3cf05945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:35.448572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1a8ae-6e1a-487d-b7c8-87ff3cf05945 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:35.449243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1a8ae-6e1a-487d-b7c8-87ff3cf05945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:17:35.449633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1a8ae-6e1a-487d-b7c8-87ff3cf05945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:35.451250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1a8ae-6e1a-487d-b7c8-87ff3cf05945 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:35.463501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1121/2255] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:17:35.467660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad159c58-b387-46b7-8ad9-b21904e5ed04 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:35.476964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad159c58-b387-46b7-8ad9-b21904e5ed04 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:35.482375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381556b-06dc-4459-a3ea-e72a2d5ba71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:35.482979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381556b-06dc-4459-a3ea-e72a2d5ba71d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:35.483300 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381556b-06dc-4459-a3ea-e72a2d5ba71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:35.483578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381556b-06dc-4459-a3ea-e72a2d5ba71d admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:35.484221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381556b-06dc-4459-a3ea-e72a2d5ba71d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:35.486157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381556b-06dc-4459-a3ea-e72a2d5ba71d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:35.514789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1135/2256] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:17:35 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:17:35.518738 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1a8ae-6e1a-487d-b7c8-87ff3cf05945 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:35.529309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1a8ae-6e1a-487d-b7c8-87ff3cf05945 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:35.586332 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebef1c-ac80-4fd8-a7d8-e960023d531c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:35.586948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebef1c-ac80-4fd8-a7d8-e960023d531c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:35.587262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebef1c-ac80-4fd8-a7d8-e960023d531c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:35.587540 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebef1c-ac80-4fd8-a7d8-e960023d531c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:35.588183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebef1c-ac80-4fd8-a7d8-e960023d531c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:35.590721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebef1c-ac80-4fd8-a7d8-e960023d531c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:35.623901 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1122/2257] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:17:35 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:17:35.627996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381556b-06dc-4459-a3ea-e72a2d5ba71d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:35.638383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381556b-06dc-4459-a3ea-e72a2d5ba71d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:35.642942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e854d-5b94-40b6-8871-fe380b58decb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:35.643562 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e854d-5b94-40b6-8871-fe380b58decb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:35.643837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e854d-5b94-40b6-8871-fe380b58decb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:35.644104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e854d-5b94-40b6-8871-fe380b58decb admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:35.644737 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e854d-5b94-40b6-8871-fe380b58decb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:35.646318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e854d-5b94-40b6-8871-fe380b58decb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:35.657424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1136/2258] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:17:35.661390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebef1c-ac80-4fd8-a7d8-e960023d531c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:35.672752 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebef1c-ac80-4fd8-a7d8-e960023d531c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:35.677614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe080b-9d37-4c56-9a16-9f98fc1a0494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:35.678213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe080b-9d37-4c56-9a16-9f98fc1a0494 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:35.678491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe080b-9d37-4c56-9a16-9f98fc1a0494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:35.678765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe080b-9d37-4c56-9a16-9f98fc1a0494 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:35.679429 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe080b-9d37-4c56-9a16-9f98fc1a0494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:35.681026 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe080b-9d37-4c56-9a16-9f98fc1a0494 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:35.691702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1123/2259] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:17:35.695964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e854d-5b94-40b6-8871-fe380b58decb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:35.705871 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e854d-5b94-40b6-8871-fe380b58decb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:35.710453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb09b0-75d8-4e4c-ac86-d43f7e16aa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:35.711133 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb09b0-75d8-4e4c-ac86-d43f7e16aa9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:35.711414 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb09b0-75d8-4e4c-ac86-d43f7e16aa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:35.711686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb09b0-75d8-4e4c-ac86-d43f7e16aa9a admin admin] PATH_INFO: `/v3/projects/345a9eb6369c43a681649f1bb55ce60c/users/41ec6426ddf640389083cc61e948d31f/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:35.728015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb09b0-75d8-4e4c-ac86-d43f7e16aa9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=345a9eb6369c43a681649f1bb55ce60c, user_id=41ec6426ddf640389083cc61e948d31f, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:35.731315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb09b0-75d8-4e4c-ac86-d43f7e16aa9a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:35.744307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1137/2260] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:17:35 2022] PUT /identity/v3/projects/345a9eb6369c43a681649f1bb55ce60c/users/41ec6426ddf640389083cc61e948d31f/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:17:35.749330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a9a44-46d9-4245-90fd-d8d469f5e059 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:35.749809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a9a44-46d9-4245-90fd-d8d469f5e059 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:35.749901 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a9a44-46d9-4245-90fd-d8d469f5e059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:35.768617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c72a9a44-46d9-4245-90fd-d8d469f5e059 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:35.769806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c72a9a44-46d9-4245-90fd-d8d469f5e059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:35.841301 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1124/2261] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:35.851221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb09b0-75d8-4e4c-ac86-d43f7e16aa9a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:35.852912 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-48bb09b0-75d8-4e4c-ac86-d43f7e16aa9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:35.910577 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb09b0-75d8-4e4c-ac86-d43f7e16aa9a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:35.914156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbe73f-5784-4633-a3ec-13bbed86b34d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:35.914760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbe73f-5784-4633-a3ec-13bbed86b34d service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:35.915040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbe73f-5784-4633-a3ec-13bbed86b34d service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:35.915339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbe73f-5784-4633-a3ec-13bbed86b34d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:35.925053 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbe73f-5784-4633-a3ec-13bbed86b34d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:35.926468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbe73f-5784-4633-a3ec-13bbed86b34d service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:35.936001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1138/2262] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:17:36.631988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58b169-22c3-4e2a-86ea-c55d8433fa4d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:36.632365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58b169-22c3-4e2a-86ea-c55d8433fa4d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:36.632711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58b169-22c3-4e2a-86ea-c55d8433fa4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:36.661410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ee58b169-22c3-4e2a-86ea-c55d8433fa4d None None] MFA Rules not processed for user `41ec6426ddf640389083cc61e948d31f`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:36.662639 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ee58b169-22c3-4e2a-86ea-c55d8433fa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:36.703116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:17:36.736791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1125/2263] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:36.748048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbe73f-5784-4633-a3ec-13bbed86b34d service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:36.749447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-efdbe73f-5784-4633-a3ec-13bbed86b34d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:36.809837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbe73f-5784-4633-a3ec-13bbed86b34d service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:36.813600 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f161597-97af-4421-8a57-2bf153df18b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:36.814200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f161597-97af-4421-8a57-2bf153df18b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:36.814537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f161597-97af-4421-8a57-2bf153df18b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:36.814809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f161597-97af-4421-8a57-2bf153df18b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:36.823746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f161597-97af-4421-8a57-2bf153df18b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:36.825834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f161597-97af-4421-8a57-2bf153df18b0 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:36.834770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1139/2264] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:17:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:17:39.263006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60f046-8657-446c-8adf-f4deec16846d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:39.263548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60f046-8657-446c-8adf-f4deec16846d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:39.264014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60f046-8657-446c-8adf-f4deec16846d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:39.286439 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8b60f046-8657-446c-8adf-f4deec16846d None None] MFA Rules not processed for user `41ec6426ddf640389083cc61e948d31f`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:39.287668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8b60f046-8657-446c-8adf-f4deec16846d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:39.294995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1126/2265] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:46.830539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=85,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:17:48.575659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f161597-97af-4421-8a57-2bf153df18b0 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:48.578299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7f161597-97af-4421-8a57-2bf153df18b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:48.595889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f161597-97af-4421-8a57-2bf153df18b0 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:48.597930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:48.598369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:48.598810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:48.598931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] PATH_INFO: `/v3/users/9eb1f32b0586462f82ffd2f5e8abbd11` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:48.600416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eb1f32b0586462f82ffd2f5e8abbd11)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:48.602300 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:48.665220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eb1f32b0586462f82ffd2f5e8abbd11'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:48.669128 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eb1f32b0586462f82ffd2f5e8abbd11'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:48.669766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eb1f32b0586462f82ffd2f5e8abbd11'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:48.683124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eb1f32b0586462f82ffd2f5e8abbd11'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:48.690196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eb1f32b0586462f82ffd2f5e8abbd11'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:48.696870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1140/2266] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:17:48 2022] DELETE /identity/v3/users/9eb1f32b0586462f82ffd2f5e8abbd11 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:17:48.701975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2e73e-526a-4aca-8346-8eafb167520e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:48.702960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2e73e-526a-4aca-8346-8eafb167520e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:48.702960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2e73e-526a-4aca-8346-8eafb167520e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:48.722080 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3ee2e73e-526a-4aca-8346-8eafb167520e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:48.723348 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3ee2e73e-526a-4aca-8346-8eafb167520e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:48.788281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1127/2267] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:48.799319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:48.800821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:48.865147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db7d29-d932-47d6-bc80-428339835f53 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:48.869576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b7720-fffe-4e61-b4df-6db685b72544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:48.870165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b7720-fffe-4e61-b4df-6db685b72544 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:48.870442 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b7720-fffe-4e61-b4df-6db685b72544 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:48.871333 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b7720-fffe-4e61-b4df-6db685b72544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:48.881955 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b7720-fffe-4e61-b4df-6db685b72544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:48.883477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b7720-fffe-4e61-b4df-6db685b72544 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:48.894869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1141/2268] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:17:49.198463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2e73e-526a-4aca-8346-8eafb167520e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:49.205577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3ee2e73e-526a-4aca-8346-8eafb167520e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:49.232537 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2e73e-526a-4aca-8346-8eafb167520e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:49.240340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0baa2d4-35e9-4896-8478-009df2456fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:49.241206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0baa2d4-35e9-4896-8478-009df2456fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:49.242090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0baa2d4-35e9-4896-8478-009df2456fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:49.242763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0baa2d4-35e9-4896-8478-009df2456fb2 admin admin] PATH_INFO: `/v3/projects/72081ed54ba24a16a2e078c333c8ba07` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:49.244255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0baa2d4-35e9-4896-8478-009df2456fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72081ed54ba24a16a2e078c333c8ba07)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:49.246595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0baa2d4-35e9-4896-8478-009df2456fb2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:49.306075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c0baa2d4-35e9-4896-8478-009df2456fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72081ed54ba24a16a2e078c333c8ba07', 'request_id': 'req-c0baa2d4-35e9-4896-8478-009df2456fb2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:49.321579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c0baa2d4-35e9-4896-8478-009df2456fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72081ed54ba24a16a2e078c333c8ba07', 'request_id': 'req-c0baa2d4-35e9-4896-8478-009df2456fb2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:49.328572 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c0baa2d4-35e9-4896-8478-009df2456fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72081ed54ba24a16a2e078c333c8ba07', 'request_id': 'req-c0baa2d4-35e9-4896-8478-009df2456fb2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:49.344307 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c0baa2d4-35e9-4896-8478-009df2456fb2 admin admin] The token cache is being invalidate because project 72081ed54ba24a16a2e078c333c8ba07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:17:49.344787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c0baa2d4-35e9-4896-8478-009df2456fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:49.345538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c0baa2d4-35e9-4896-8478-009df2456fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:17:49.347176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1128/2269] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:17:49 2022] DELETE /identity/v3/projects/72081ed54ba24a16a2e078c333c8ba07 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:17:49.352384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ad041-d0ef-4edf-9503-35f298e8b4a9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:49.352719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ad041-d0ef-4edf-9503-35f298e8b4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:49.353058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ad041-d0ef-4edf-9503-35f298e8b4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:49.373667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a93ad041-d0ef-4edf-9503-35f298e8b4a9 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:49.374937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a93ad041-d0ef-4edf-9503-35f298e8b4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:49.458625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1142/2270] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:49.467018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70492fab-0274-4e64-bdef-2f270accc529 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:49.467586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70492fab-0274-4e64-bdef-2f270accc529 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:49.467862 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70492fab-0274-4e64-bdef-2f270accc529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:49.493144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-70492fab-0274-4e64-bdef-2f270accc529 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:49.494453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-70492fab-0274-4e64-bdef-2f270accc529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:49.504220 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1129/2271] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:49.509122 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ad041-d0ef-4edf-9503-35f298e8b4a9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:49.520582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ad041-d0ef-4edf-9503-35f298e8b4a9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:49.527736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23158b8-c5df-4b8e-9848-166b97ac2911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:49.528413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23158b8-c5df-4b8e-9848-166b97ac2911 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:49.528785 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23158b8-c5df-4b8e-9848-166b97ac2911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:49.529007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23158b8-c5df-4b8e-9848-166b97ac2911 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:49.529665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23158b8-c5df-4b8e-9848-166b97ac2911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:17:49.530032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23158b8-c5df-4b8e-9848-166b97ac2911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:49.531722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23158b8-c5df-4b8e-9848-166b97ac2911 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:49.549011 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1143/2272] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:17:49.551421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70492fab-0274-4e64-bdef-2f270accc529 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:49.574531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70492fab-0274-4e64-bdef-2f270accc529 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:49.581073 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831073c6-faa8-4db2-a5c5-06ab27c82f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:49.582071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831073c6-faa8-4db2-a5c5-06ab27c82f1b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:49.582589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831073c6-faa8-4db2-a5c5-06ab27c82f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:49.582834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831073c6-faa8-4db2-a5c5-06ab27c82f1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:49.583643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831073c6-faa8-4db2-a5c5-06ab27c82f1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:49.586597 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831073c6-faa8-4db2-a5c5-06ab27c82f1b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:49.642391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1130/2273] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:17:49 2022] POST /identity/v3/projects => generated 399 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:17:49.643650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23158b8-c5df-4b8e-9848-166b97ac2911 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:49.650192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:17:49.659006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23158b8-c5df-4b8e-9848-166b97ac2911 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:49.733166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96bf2c-f6ae-44ae-8fda-8a2004fc1bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:49.733907 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96bf2c-f6ae-44ae-8fda-8a2004fc1bfb admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:49.734370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96bf2c-f6ae-44ae-8fda-8a2004fc1bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:49.734739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96bf2c-f6ae-44ae-8fda-8a2004fc1bfb admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:49.735550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96bf2c-f6ae-44ae-8fda-8a2004fc1bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:49.737590 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96bf2c-f6ae-44ae-8fda-8a2004fc1bfb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:49.771982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1144/2274] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:17:49 2022] POST /identity/v3/users => generated 300 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:17:49.777714 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831073c6-faa8-4db2-a5c5-06ab27c82f1b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:49.791755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831073c6-faa8-4db2-a5c5-06ab27c82f1b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:49.806137 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a124f8-70ef-44a8-bfb8-e72f2b5f0128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:49.806790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a124f8-70ef-44a8-bfb8-e72f2b5f0128 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:49.807123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a124f8-70ef-44a8-bfb8-e72f2b5f0128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:49.807453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a124f8-70ef-44a8-bfb8-e72f2b5f0128 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:49.808884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a124f8-70ef-44a8-bfb8-e72f2b5f0128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:49.810949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a124f8-70ef-44a8-bfb8-e72f2b5f0128 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:49.822783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1131/2275] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:17:49.826587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96bf2c-f6ae-44ae-8fda-8a2004fc1bfb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:49.842135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96bf2c-f6ae-44ae-8fda-8a2004fc1bfb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:49.850067 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a3fdf-53bb-493f-b638-6f0491c387f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:49.850705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a3fdf-53bb-493f-b638-6f0491c387f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:49.850991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a3fdf-53bb-493f-b638-6f0491c387f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:49.851464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a3fdf-53bb-493f-b638-6f0491c387f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:49.851986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a3fdf-53bb-493f-b638-6f0491c387f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:49.854325 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a3fdf-53bb-493f-b638-6f0491c387f0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:49.870441 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1145/2276] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:17:49.871880 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a124f8-70ef-44a8-bfb8-e72f2b5f0128 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:49.884066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a124f8-70ef-44a8-bfb8-e72f2b5f0128 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:49.893920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebc852-6a0f-48b6-af07-820f5f5661e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:49.895658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebc852-6a0f-48b6-af07-820f5f5661e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:49.896044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebc852-6a0f-48b6-af07-820f5f5661e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:49.896365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebc852-6a0f-48b6-af07-820f5f5661e8 admin admin] PATH_INFO: `/v3/projects/a70e7eb54cf24c06b7265b841c1cb1de/users/a39ba64ce6a44f2da6ad04b13752830a/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:49.915677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:17:49.919330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebc852-6a0f-48b6-af07-820f5f5661e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a70e7eb54cf24c06b7265b841c1cb1de, user_id=a39ba64ce6a44f2da6ad04b13752830a, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:49.922397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebc852-6a0f-48b6-af07-820f5f5661e8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:49.937668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1132/2277] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:17:49 2022] PUT /identity/v3/projects/a70e7eb54cf24c06b7265b841c1cb1de/users/a39ba64ce6a44f2da6ad04b13752830a/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:17:49.948312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbace2-8e3d-4cb2-b28a-138c88ba9413 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:49.948733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbace2-8e3d-4cb2-b28a-138c88ba9413 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:49.949080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbace2-8e3d-4cb2-b28a-138c88ba9413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:49.978271 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-38cbace2-8e3d-4cb2-b28a-138c88ba9413 None None] MFA Rules not processed for user `a39ba64ce6a44f2da6ad04b13752830a`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:17:49.979662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-38cbace2-8e3d-4cb2-b28a-138c88ba9413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:50.048451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1146/2278] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:17:49 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:17:50.060579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebc852-6a0f-48b6-af07-820f5f5661e8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:50.062983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0eebc852-6a0f-48b6-af07-820f5f5661e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:50.122581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebc852-6a0f-48b6-af07-820f5f5661e8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:50.126410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1597d4f-a2da-4661-8eb5-3379c9bd6c08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:50.127028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1597d4f-a2da-4661-8eb5-3379c9bd6c08 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:50.127339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1597d4f-a2da-4661-8eb5-3379c9bd6c08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:50.127617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1597d4f-a2da-4661-8eb5-3379c9bd6c08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:50.136584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1597d4f-a2da-4661-8eb5-3379c9bd6c08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:50.138112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1597d4f-a2da-4661-8eb5-3379c9bd6c08 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:50.150789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1133/2279] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:17:50 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:17:52.446121 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbace2-8e3d-4cb2-b28a-138c88ba9413 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:17:52.447741 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-38cbace2-8e3d-4cb2-b28a-138c88ba9413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:52.534487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbace2-8e3d-4cb2-b28a-138c88ba9413 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:17:52.539934 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ef2f0-264b-427b-b8d4-6111f2a084f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:17:52.540671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ef2f0-264b-427b-b8d4-6111f2a084f0 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:17:52.541035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ef2f0-264b-427b-b8d4-6111f2a084f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:17:52.541371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ef2f0-264b-427b-b8d4-6111f2a084f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:17:52.543423 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d92ef2f0-264b-427b-b8d4-6111f2a084f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:17:52.556787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ef2f0-264b-427b-b8d4-6111f2a084f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:17:52.558704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ef2f0-264b-427b-b8d4-6111f2a084f0 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:17:52.565921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:17:52.572228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1147/2280] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:17:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:18:00.143253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=29,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:01.511137 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1597d4f-a2da-4661-8eb5-3379c9bd6c08 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:01.578290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1597d4f-a2da-4661-8eb5-3379c9bd6c08 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:01.583475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:01.584257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:01.584654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:01.585017 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] PATH_INFO: `/v3/users/a39ba64ce6a44f2da6ad04b13752830a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:01.586252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a39ba64ce6a44f2da6ad04b13752830a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:01.588339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:01.649406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a39ba64ce6a44f2da6ad04b13752830a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:01.658471 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a39ba64ce6a44f2da6ad04b13752830a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:01.673480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a39ba64ce6a44f2da6ad04b13752830a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:01.674308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a39ba64ce6a44f2da6ad04b13752830a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:01.682655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a39ba64ce6a44f2da6ad04b13752830a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:01.689550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1134/2281] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:18:01 2022] DELETE /identity/v3/users/a39ba64ce6a44f2da6ad04b13752830a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:18:01.693979 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf357a9-573a-4773-8180-435e5708cfc0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:01.694789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf357a9-573a-4773-8180-435e5708cfc0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:01.694789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf357a9-573a-4773-8180-435e5708cfc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:01.717851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ccf357a9-573a-4773-8180-435e5708cfc0 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:01.719236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ccf357a9-573a-4773-8180-435e5708cfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:01.782661 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1148/2282] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:01.792914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:01.795209 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:01.859121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f39a2a-d726-47b9-8768-fb72e0e3dffb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:01.863250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b1f9f-cc84-4c18-bd52-e902c81e5ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:01.875284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b1f9f-cc84-4c18-bd52-e902c81e5ffc service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:01.875284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b1f9f-cc84-4c18-bd52-e902c81e5ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:01.875284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b1f9f-cc84-4c18-bd52-e902c81e5ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:01.875284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b1f9f-cc84-4c18-bd52-e902c81e5ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:01.876583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b1f9f-cc84-4c18-bd52-e902c81e5ffc service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:01.885948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1135/2283] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:18:02.164074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf357a9-573a-4773-8180-435e5708cfc0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:02.165633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ccf357a9-573a-4773-8180-435e5708cfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:02.178204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf357a9-573a-4773-8180-435e5708cfc0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:02.182986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:02.183754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:02.184068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:02.184255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a admin admin] PATH_INFO: `/v3/projects/a70e7eb54cf24c06b7265b841c1cb1de` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:02.185408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a70e7eb54cf24c06b7265b841c1cb1de)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:02.187356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:02.237560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a70e7eb54cf24c06b7265b841c1cb1de', 'request_id': 'req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:02.245508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a70e7eb54cf24c06b7265b841c1cb1de', 'request_id': 'req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:02.251469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a70e7eb54cf24c06b7265b841c1cb1de', 'request_id': 'req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:02.263261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a admin admin] The token cache is being invalidate because project a70e7eb54cf24c06b7265b841c1cb1de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:18:02.263551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:02.264214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2857f5fe-bb71-47c1-ae9e-d6e07f3b963a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:02.265723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1149/2284] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:18:02 2022] DELETE /identity/v3/projects/a70e7eb54cf24c06b7265b841c1cb1de => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:18:02.271137 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe4eb2-111a-4965-9481-a5ef0f791dcd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:02.271430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe4eb2-111a-4965-9481-a5ef0f791dcd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:02.271709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe4eb2-111a-4965-9481-a5ef0f791dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:02.291336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7bfe4eb2-111a-4965-9481-a5ef0f791dcd None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:02.292576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7bfe4eb2-111a-4965-9481-a5ef0f791dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:02.309672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:02.355647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1136/2285] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:02.363035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bb721-e50a-4a4d-a277-7c99ccb817a9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:02.363361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bb721-e50a-4a4d-a277-7c99ccb817a9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:02.363641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bb721-e50a-4a4d-a277-7c99ccb817a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:02.383329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-947bb721-e50a-4a4d-a277-7c99ccb817a9 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:02.384540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-947bb721-e50a-4a4d-a277-7c99ccb817a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:02.393140 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1150/2286] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:02.397332 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe4eb2-111a-4965-9481-a5ef0f791dcd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:02.407920 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe4eb2-111a-4965-9481-a5ef0f791dcd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:02.412949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a545391-229c-436c-91f1-27057c22896f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:02.413582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a545391-229c-436c-91f1-27057c22896f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:02.413855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a545391-229c-436c-91f1-27057c22896f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:02.414125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a545391-229c-436c-91f1-27057c22896f admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:02.414800 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a545391-229c-436c-91f1-27057c22896f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:18:02.415129 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a545391-229c-436c-91f1-27057c22896f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:02.416764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a545391-229c-436c-91f1-27057c22896f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:02.432230 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1137/2287] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:18:02.433398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bb721-e50a-4a4d-a277-7c99ccb817a9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:02.433726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a545391-229c-436c-91f1-27057c22896f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:02.435441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9a545391-229c-436c-91f1-27057c22896f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:02.442601 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bb721-e50a-4a4d-a277-7c99ccb817a9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:02.447277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b16a51-f7fe-4c29-9853-ee7d6de43e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:02.447880 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b16a51-f7fe-4c29-9853-ee7d6de43e49 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:02.448154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b16a51-f7fe-4c29-9853-ee7d6de43e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:02.448423 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b16a51-f7fe-4c29-9853-ee7d6de43e49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:02.449047 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a545391-229c-436c-91f1-27057c22896f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:02.449768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b16a51-f7fe-4c29-9853-ee7d6de43e49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:02.451682 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b16a51-f7fe-4c29-9853-ee7d6de43e49 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:02.454141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca514380-a658-4bfd-b4ce-2385c656ecd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:02.454854 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca514380-a658-4bfd-b4ce-2385c656ecd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:02.455253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca514380-a658-4bfd-b4ce-2385c656ecd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:02.455625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca514380-a658-4bfd-b4ce-2385c656ecd3 admin admin] PATH_INFO: `/v3/users/9824d4e028f940a4bdd15f9f56aa8299` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:02.456895 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca514380-a658-4bfd-b4ce-2385c656ecd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9824d4e028f940a4bdd15f9f56aa8299)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:02.459077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca514380-a658-4bfd-b4ce-2385c656ecd3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:02.485742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1151/2288] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:18:02 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:18:02.487111 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b16a51-f7fe-4c29-9853-ee7d6de43e49 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:02.514142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ca514380-a658-4bfd-b4ce-2385c656ecd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9824d4e028f940a4bdd15f9f56aa8299'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:02.522118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ca514380-a658-4bfd-b4ce-2385c656ecd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9824d4e028f940a4bdd15f9f56aa8299'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:02.534623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ca514380-a658-4bfd-b4ce-2385c656ecd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9824d4e028f940a4bdd15f9f56aa8299'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:02.535307 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ca514380-a658-4bfd-b4ce-2385c656ecd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9824d4e028f940a4bdd15f9f56aa8299'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:02.542186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ca514380-a658-4bfd-b4ce-2385c656ecd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9824d4e028f940a4bdd15f9f56aa8299'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:02.548554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1138/2289] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:18:02 2022] DELETE /identity/v3/users/9824d4e028f940a4bdd15f9f56aa8299 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:18:02.548708 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:02.554472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b747d-7d1f-4ae8-ae4f-397a82a45176 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:02.554753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b747d-7d1f-4ae8-ae4f-397a82a45176 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:02.555025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b747d-7d1f-4ae8-ae4f-397a82a45176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:02.555463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b16a51-f7fe-4c29-9853-ee7d6de43e49 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:02.575651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-687b747d-7d1f-4ae8-ae4f-397a82a45176 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:02.577587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-687b747d-7d1f-4ae8-ae4f-397a82a45176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:02.622296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20078e75-231b-42e2-b699-c2f64a77d1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:02.622919 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20078e75-231b-42e2-b699-c2f64a77d1ae admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:02.623240 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20078e75-231b-42e2-b699-c2f64a77d1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:02.623517 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20078e75-231b-42e2-b699-c2f64a77d1ae admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:02.624156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20078e75-231b-42e2-b699-c2f64a77d1ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:02.626109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20078e75-231b-42e2-b699-c2f64a77d1ae admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:02.644212 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1139/2290] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:02.650784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b747d-7d1f-4ae8-ae4f-397a82a45176 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:02.652860 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-687b747d-7d1f-4ae8-ae4f-397a82a45176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:02.661857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1152/2291] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:18:02 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:18:02.663219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20078e75-231b-42e2-b699-c2f64a77d1ae admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:02.665093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-20078e75-231b-42e2-b699-c2f64a77d1ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:02.677874 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20078e75-231b-42e2-b699-c2f64a77d1ae admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:02.682173 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:02.682646 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c886875-a199-4939-a45e-0593df2bbd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:02.683314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c886875-a199-4939-a45e-0593df2bbd63 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:02.683591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c886875-a199-4939-a45e-0593df2bbd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:02.683941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c886875-a199-4939-a45e-0593df2bbd63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:02.684572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c886875-a199-4939-a45e-0593df2bbd63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:02.686296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c886875-a199-4939-a45e-0593df2bbd63 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:02.696349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1153/2292] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:18:02.700288 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c886875-a199-4939-a45e-0593df2bbd63 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:02.710667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c886875-a199-4939-a45e-0593df2bbd63 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:02.717143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cafb5-3ddc-4fd8-9515-139caa8c28f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:02.717430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b747d-7d1f-4ae8-ae4f-397a82a45176 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:02.717746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cafb5-3ddc-4fd8-9515-139caa8c28f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:02.718025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cafb5-3ddc-4fd8-9515-139caa8c28f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:02.718298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cafb5-3ddc-4fd8-9515-139caa8c28f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:02.718945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cafb5-3ddc-4fd8-9515-139caa8c28f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:02.720567 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cafb5-3ddc-4fd8-9515-139caa8c28f7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:02.721312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbc5a6-d43f-425e-873d-f1dad6ae317b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:02.721930 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbc5a6-d43f-425e-873d-f1dad6ae317b service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:02.722216 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbc5a6-d43f-425e-873d-f1dad6ae317b service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:02.722486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbc5a6-d43f-425e-873d-f1dad6ae317b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:02.733454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbc5a6-d43f-425e-873d-f1dad6ae317b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:02.734935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbc5a6-d43f-425e-873d-f1dad6ae317b service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:02.741216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1154/2293] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:18:02.745887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cafb5-3ddc-4fd8-9515-139caa8c28f7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:02.748411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1140/2294] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:18:02.757956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cafb5-3ddc-4fd8-9515-139caa8c28f7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:02.764132 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d963b-a46d-4b5a-8b6f-8e010c67fc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:02.764851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d963b-a46d-4b5a-8b6f-8e010c67fc7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:02.765112 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d963b-a46d-4b5a-8b6f-8e010c67fc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:02.765386 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d963b-a46d-4b5a-8b6f-8e010c67fc7e admin admin] PATH_INFO: `/v3/projects/e7c4b6192be646b38d7757960e02feda/users/b754de037a8148358efb03ffa39786d4/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:02.783151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d963b-a46d-4b5a-8b6f-8e010c67fc7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c4b6192be646b38d7757960e02feda, user_id=b754de037a8148358efb03ffa39786d4, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:02.786311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d963b-a46d-4b5a-8b6f-8e010c67fc7e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:02.803239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1155/2295] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:18:02 2022] PUT /identity/v3/projects/e7c4b6192be646b38d7757960e02feda/users/b754de037a8148358efb03ffa39786d4/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:18:02.809485 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be3d50-6a09-4bf2-a117-1bd833585420 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:02.809775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be3d50-6a09-4bf2-a117-1bd833585420 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:02.810053 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be3d50-6a09-4bf2-a117-1bd833585420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:02.832853 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-31be3d50-6a09-4bf2-a117-1bd833585420 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:02.834393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-31be3d50-6a09-4bf2-a117-1bd833585420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:02.911093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1141/2296] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:02.932018 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d963b-a46d-4b5a-8b6f-8e010c67fc7e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:02.934834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0e3d963b-a46d-4b5a-8b6f-8e010c67fc7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:03.015032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d963b-a46d-4b5a-8b6f-8e010c67fc7e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:03.019713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e1fb5-8f5a-4c42-b4b3-737db2201d3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:03.020485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e1fb5-8f5a-4c42-b4b3-737db2201d3d service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:03.020881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e1fb5-8f5a-4c42-b4b3-737db2201d3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:03.021222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e1fb5-8f5a-4c42-b4b3-737db2201d3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:03.037076 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e1fb5-8f5a-4c42-b4b3-737db2201d3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:03.038674 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e1fb5-8f5a-4c42-b4b3-737db2201d3d service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:03.052866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1156/2297] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:18:03.230856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be3d50-6a09-4bf2-a117-1bd833585420 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:03.232444 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-31be3d50-6a09-4bf2-a117-1bd833585420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:03.244783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be3d50-6a09-4bf2-a117-1bd833585420 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:03.250339 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6973ec-c7ad-43d7-ae47-3a9e35381221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:03.251077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6973ec-c7ad-43d7-ae47-3a9e35381221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:03.251357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6973ec-c7ad-43d7-ae47-3a9e35381221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:03.251629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6973ec-c7ad-43d7-ae47-3a9e35381221 admin admin] PATH_INFO: `/v3/projects/e6f3b1b36a4b441b914525415c19e9ba` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:03.252938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6973ec-c7ad-43d7-ae47-3a9e35381221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6f3b1b36a4b441b914525415c19e9ba)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:03.255543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6973ec-c7ad-43d7-ae47-3a9e35381221 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:03.313508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9d6973ec-c7ad-43d7-ae47-3a9e35381221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6f3b1b36a4b441b914525415c19e9ba', 'request_id': 'req-9d6973ec-c7ad-43d7-ae47-3a9e35381221'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:03.320299 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9d6973ec-c7ad-43d7-ae47-3a9e35381221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6f3b1b36a4b441b914525415c19e9ba', 'request_id': 'req-9d6973ec-c7ad-43d7-ae47-3a9e35381221'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:03.329284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9d6973ec-c7ad-43d7-ae47-3a9e35381221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6f3b1b36a4b441b914525415c19e9ba', 'request_id': 'req-9d6973ec-c7ad-43d7-ae47-3a9e35381221'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:03.343717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9d6973ec-c7ad-43d7-ae47-3a9e35381221 admin admin] The token cache is being invalidate because project e6f3b1b36a4b441b914525415c19e9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:18:03.344144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9d6973ec-c7ad-43d7-ae47-3a9e35381221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:03.345227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9d6973ec-c7ad-43d7-ae47-3a9e35381221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:03.346799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1142/2298] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:18:03 2022] DELETE /identity/v3/projects/e6f3b1b36a4b441b914525415c19e9ba => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:18:05.270300 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e1fb5-8f5a-4c42-b4b3-737db2201d3d service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:05.271837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-275e1fb5-8f5a-4c42-b4b3-737db2201d3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:05.285349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e1fb5-8f5a-4c42-b4b3-737db2201d3d service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:05.348803 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:05.368131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:05.368794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:05.369076 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:05.369351 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] PATH_INFO: `/v3/users/0dd9f9a08c02453e8a5a8e40ebf72b3a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:05.371483 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dd9f9a08c02453e8a5a8e40ebf72b3a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:05.373670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:05.433527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dd9f9a08c02453e8a5a8e40ebf72b3a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:05.442261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dd9f9a08c02453e8a5a8e40ebf72b3a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:05.442940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dd9f9a08c02453e8a5a8e40ebf72b3a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:05.457910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dd9f9a08c02453e8a5a8e40ebf72b3a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:05.466013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dd9f9a08c02453e8a5a8e40ebf72b3a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:05.472643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1157/2299] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:18:05 2022] DELETE /identity/v3/users/0dd9f9a08c02453e8a5a8e40ebf72b3a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:18:05.477006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcc604-c757-4bcd-a354-2748e7c0b427 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:05.477544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcc604-c757-4bcd-a354-2748e7c0b427 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:05.477544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcc604-c757-4bcd-a354-2748e7c0b427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:05.499309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-44dcc604-c757-4bcd-a354-2748e7c0b427 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:05.500523 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-44dcc604-c757-4bcd-a354-2748e7c0b427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:05.560504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:05.585192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1143/2300] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:05.603795 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:05.613136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:05.684419 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce1703-8bef-4f23-a6ae-6938f121561f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:05.688448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c18d8-61b6-4ad8-9c48-46d8f4797d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:05.689226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c18d8-61b6-4ad8-9c48-46d8f4797d8e service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:05.689507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c18d8-61b6-4ad8-9c48-46d8f4797d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:05.689775 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c18d8-61b6-4ad8-9c48-46d8f4797d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:05.700222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c18d8-61b6-4ad8-9c48-46d8f4797d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:05.701673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c18d8-61b6-4ad8-9c48-46d8f4797d8e service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:05.712949 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1158/2301] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:18:06.084252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcc604-c757-4bcd-a354-2748e7c0b427 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:06.086150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-44dcc604-c757-4bcd-a354-2748e7c0b427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:06.101106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcc604-c757-4bcd-a354-2748e7c0b427 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:06.107371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424eb55-83ef-4e47-9891-d2ce269ad8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:06.108235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424eb55-83ef-4e47-9891-d2ce269ad8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:06.108359 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424eb55-83ef-4e47-9891-d2ce269ad8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:06.108539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424eb55-83ef-4e47-9891-d2ce269ad8fd admin admin] PATH_INFO: `/v3/projects/b4a91ae8f1b54a89b153b4c6030ff455` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:06.110102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424eb55-83ef-4e47-9891-d2ce269ad8fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a91ae8f1b54a89b153b4c6030ff455)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:06.111712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424eb55-83ef-4e47-9891-d2ce269ad8fd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:06.176219 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9424eb55-83ef-4e47-9891-d2ce269ad8fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a91ae8f1b54a89b153b4c6030ff455', 'request_id': 'req-9424eb55-83ef-4e47-9891-d2ce269ad8fd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:06.182501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9424eb55-83ef-4e47-9891-d2ce269ad8fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a91ae8f1b54a89b153b4c6030ff455', 'request_id': 'req-9424eb55-83ef-4e47-9891-d2ce269ad8fd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:06.190028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9424eb55-83ef-4e47-9891-d2ce269ad8fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a91ae8f1b54a89b153b4c6030ff455', 'request_id': 'req-9424eb55-83ef-4e47-9891-d2ce269ad8fd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:06.204890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9424eb55-83ef-4e47-9891-d2ce269ad8fd admin admin] The token cache is being invalidate because project b4a91ae8f1b54a89b153b4c6030ff455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:18:06.205460 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9424eb55-83ef-4e47-9891-d2ce269ad8fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:06.210535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9424eb55-83ef-4e47-9891-d2ce269ad8fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:18:06.214441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1144/2302] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:18:06 2022] DELETE /identity/v3/projects/b4a91ae8f1b54a89b153b4c6030ff455 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:18:06.215595 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a1f43-0277-4ad4-8e82-da014a813a2b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:06.215883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a1f43-0277-4ad4-8e82-da014a813a2b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:06.216158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a1f43-0277-4ad4-8e82-da014a813a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:06.262181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8f0a1f43-0277-4ad4-8e82-da014a813a2b None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:06.263782 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8f0a1f43-0277-4ad4-8e82-da014a813a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:06.367757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7063b2-1313-463f-a4ab-396109603ace None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:06.368142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7063b2-1313-463f-a4ab-396109603ace None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:06.368544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7063b2-1313-463f-a4ab-396109603ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:06.371533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1159/2303] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:06.398237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3a7063b2-1313-463f-a4ab-396109603ace None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:06.399618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3a7063b2-1313-463f-a4ab-396109603ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:06.413330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1145/2304] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:06.414368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a1f43-0277-4ad4-8e82-da014a813a2b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:06.436345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a1f43-0277-4ad4-8e82-da014a813a2b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:06.453107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de023e03-b523-46c2-80f7-6dad311b80d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:06.454051 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de023e03-b523-46c2-80f7-6dad311b80d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:06.454434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de023e03-b523-46c2-80f7-6dad311b80d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:06.454737 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de023e03-b523-46c2-80f7-6dad311b80d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:06.455458 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de023e03-b523-46c2-80f7-6dad311b80d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:18:06.455766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de023e03-b523-46c2-80f7-6dad311b80d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:06.465055 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de023e03-b523-46c2-80f7-6dad311b80d8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:06.474877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:06.490246 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1160/2305] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:18:06.490968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7063b2-1313-463f-a4ab-396109603ace None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:06.505795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7063b2-1313-463f-a4ab-396109603ace None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:06.522535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926dd349-afa6-4ab8-89d2-cbe2a6de3e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:06.523254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926dd349-afa6-4ab8-89d2-cbe2a6de3e6b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:06.523587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926dd349-afa6-4ab8-89d2-cbe2a6de3e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:06.523863 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926dd349-afa6-4ab8-89d2-cbe2a6de3e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:06.527624 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926dd349-afa6-4ab8-89d2-cbe2a6de3e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:06.537345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926dd349-afa6-4ab8-89d2-cbe2a6de3e6b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:06.612236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1146/2306] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:18:06 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:18:06.617854 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de023e03-b523-46c2-80f7-6dad311b80d8 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:06.638192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de023e03-b523-46c2-80f7-6dad311b80d8 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:06.709805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80798bdf-b97e-4597-8d4b-e7b065d19170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:06.710607 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80798bdf-b97e-4597-8d4b-e7b065d19170 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:06.710993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80798bdf-b97e-4597-8d4b-e7b065d19170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:06.711403 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80798bdf-b97e-4597-8d4b-e7b065d19170 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:06.712154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80798bdf-b97e-4597-8d4b-e7b065d19170 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:06.714247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80798bdf-b97e-4597-8d4b-e7b065d19170 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:06.754741 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1161/2307] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:18:06 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:18:06.756036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926dd349-afa6-4ab8-89d2-cbe2a6de3e6b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:06.768813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926dd349-afa6-4ab8-89d2-cbe2a6de3e6b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:06.774604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc19738-ac0a-4923-ae72-55d930143b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:06.775659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc19738-ac0a-4923-ae72-55d930143b97 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:06.776046 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc19738-ac0a-4923-ae72-55d930143b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:06.776452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc19738-ac0a-4923-ae72-55d930143b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:06.777375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc19738-ac0a-4923-ae72-55d930143b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:06.779124 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc19738-ac0a-4923-ae72-55d930143b97 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:06.793910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1147/2308] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:18:06.795250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80798bdf-b97e-4597-8d4b-e7b065d19170 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:06.806447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80798bdf-b97e-4597-8d4b-e7b065d19170 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:06.812833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc9f7b-6af9-4692-a780-38a7c7e34874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:06.813566 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc9f7b-6af9-4692-a780-38a7c7e34874 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:06.814294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc9f7b-6af9-4692-a780-38a7c7e34874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:06.814294 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc9f7b-6af9-4692-a780-38a7c7e34874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:06.815032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc9f7b-6af9-4692-a780-38a7c7e34874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:06.817485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc9f7b-6af9-4692-a780-38a7c7e34874 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:06.830084 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1162/2309] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:18:06.839753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc19738-ac0a-4923-ae72-55d930143b97 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:06.852358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc19738-ac0a-4923-ae72-55d930143b97 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:06.860499 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d7419-28d6-4842-a44d-17f37b406e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:06.862188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d7419-28d6-4842-a44d-17f37b406e87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:06.862570 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d7419-28d6-4842-a44d-17f37b406e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:06.862961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d7419-28d6-4842-a44d-17f37b406e87 admin admin] PATH_INFO: `/v3/projects/4cc8da7b799a4bf49ce080e37a36c1d0/users/3edc8b9046ac4112a838a9c3b39aecf0/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:06.889011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d7419-28d6-4842-a44d-17f37b406e87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cc8da7b799a4bf49ce080e37a36c1d0, user_id=3edc8b9046ac4112a838a9c3b39aecf0, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:06.893594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d7419-28d6-4842-a44d-17f37b406e87 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:06.926248 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1148/2310] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:18:06 2022] PUT /identity/v3/projects/4cc8da7b799a4bf49ce080e37a36c1d0/users/3edc8b9046ac4112a838a9c3b39aecf0/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:18:06.933072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99939a-5014-4342-a156-54c227f83d4c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:06.933439 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99939a-5014-4342-a156-54c227f83d4c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:06.933841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99939a-5014-4342-a156-54c227f83d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:06.959375 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4e99939a-5014-4342-a156-54c227f83d4c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:06.961029 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4e99939a-5014-4342-a156-54c227f83d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:07.067766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1163/2311] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:07.085408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d7419-28d6-4842-a44d-17f37b406e87 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:07.087567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-273d7419-28d6-4842-a44d-17f37b406e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:07.175255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d7419-28d6-4842-a44d-17f37b406e87 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:07.179213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf9a8a-4345-4b91-9272-26ed38775597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:07.179833 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf9a8a-4345-4b91-9272-26ed38775597 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:07.180136 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf9a8a-4345-4b91-9272-26ed38775597 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:07.180390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf9a8a-4345-4b91-9272-26ed38775597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:07.194345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf9a8a-4345-4b91-9272-26ed38775597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:07.195436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf9a8a-4345-4b91-9272-26ed38775597 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:07.208297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1149/2312] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:18:08.217491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8105e-6cf4-4fd6-b5b4-61e7a79b5032 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:08.217973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8105e-6cf4-4fd6-b5b4-61e7a79b5032 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:08.218055 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8105e-6cf4-4fd6-b5b4-61e7a79b5032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:08.283597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1da8105e-6cf4-4fd6-b5b4-61e7a79b5032 None None] MFA Rules not processed for user `b754de037a8148358efb03ffa39786d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:08.287119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1da8105e-6cf4-4fd6-b5b4-61e7a79b5032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:08.374161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1164/2313] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:08.378797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf9a8a-4345-4b91-9272-26ed38775597 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:08.380406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d5bf9a8a-4345-4b91-9272-26ed38775597 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:08.397791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf9a8a-4345-4b91-9272-26ed38775597 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:08.404545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88629e-fdf7-428c-8f88-321d5cbe95c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:08.406089 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88629e-fdf7-428c-8f88-321d5cbe95c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:08.406628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88629e-fdf7-428c-8f88-321d5cbe95c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:08.407140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88629e-fdf7-428c-8f88-321d5cbe95c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:08.408070 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88629e-fdf7-428c-8f88-321d5cbe95c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:08.410628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88629e-fdf7-428c-8f88-321d5cbe95c1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:08.429795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:08.466030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1150/2314] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:18:08 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:18:08.470429 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da8105e-6cf4-4fd6-b5b4-61e7a79b5032 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:08.520665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da8105e-6cf4-4fd6-b5b4-61e7a79b5032 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:08.529100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:08.631854 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd258d68-7d57-4287-a86a-7e3787094a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:08.632655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd258d68-7d57-4287-a86a-7e3787094a2d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:08.632920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd258d68-7d57-4287-a86a-7e3787094a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:08.633262 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd258d68-7d57-4287-a86a-7e3787094a2d admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:08.633959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd258d68-7d57-4287-a86a-7e3787094a2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:08.636787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd258d68-7d57-4287-a86a-7e3787094a2d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:08.667345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1165/2315] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:18:08 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:18:08.669027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88629e-fdf7-428c-8f88-321d5cbe95c1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:08.681751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88629e-fdf7-428c-8f88-321d5cbe95c1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:08.693151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0098b8-0e0a-4968-8713-2cd53faec150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:08.694249 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0098b8-0e0a-4968-8713-2cd53faec150 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:08.694249 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0098b8-0e0a-4968-8713-2cd53faec150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:08.694432 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0098b8-0e0a-4968-8713-2cd53faec150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:08.695084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0098b8-0e0a-4968-8713-2cd53faec150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:08.696822 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0098b8-0e0a-4968-8713-2cd53faec150 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:08.712654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1151/2316] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:18:08.714800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd258d68-7d57-4287-a86a-7e3787094a2d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:08.732217 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd258d68-7d57-4287-a86a-7e3787094a2d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:08.742873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cccb9-d4b9-4e3c-9fed-886599c8b7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:08.743684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cccb9-d4b9-4e3c-9fed-886599c8b7ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:08.743974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cccb9-d4b9-4e3c-9fed-886599c8b7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:08.746589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cccb9-d4b9-4e3c-9fed-886599c8b7ab admin admin] PATH_INFO: `/v3/projects/164df8b44ce844e58f7d168409990608/users/d8eee1ec9ecd4dfbae1864be468894f1/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:08.773236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689cccb9-d4b9-4e3c-9fed-886599c8b7ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=164df8b44ce844e58f7d168409990608, user_id=d8eee1ec9ecd4dfbae1864be468894f1, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:08.776660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689cccb9-d4b9-4e3c-9fed-886599c8b7ab admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:08.799832 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1166/2317] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:18:08 2022] PUT /identity/v3/projects/164df8b44ce844e58f7d168409990608/users/d8eee1ec9ecd4dfbae1864be468894f1/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:18:08.801343 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0098b8-0e0a-4968-8713-2cd53faec150 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:08.827814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0098b8-0e0a-4968-8713-2cd53faec150 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:08.970583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653c1ac-f291-4e50-af11-95914c0a206e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:08.971298 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653c1ac-f291-4e50-af11-95914c0a206e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:08.971589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653c1ac-f291-4e50-af11-95914c0a206e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:08.972355 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653c1ac-f291-4e50-af11-95914c0a206e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:08.972866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653c1ac-f291-4e50-af11-95914c0a206e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:08.975177 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653c1ac-f291-4e50-af11-95914c0a206e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:08.999225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1152/2318] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:18:09.000663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cccb9-d4b9-4e3c-9fed-886599c8b7ab admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:09.021078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cccb9-d4b9-4e3c-9fed-886599c8b7ab admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:09.031638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdc73e-beb4-497d-b09f-b1a62b0593e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:09.032436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdc73e-beb4-497d-b09f-b1a62b0593e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:09.032815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdc73e-beb4-497d-b09f-b1a62b0593e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:09.033163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdc73e-beb4-497d-b09f-b1a62b0593e0 admin admin] PATH_INFO: `/v3/projects/164df8b44ce844e58f7d168409990608/users/d8eee1ec9ecd4dfbae1864be468894f1/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:09.035591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdc73e-beb4-497d-b09f-b1a62b0593e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=164df8b44ce844e58f7d168409990608, user_id=d8eee1ec9ecd4dfbae1864be468894f1, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:09.038412 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdc73e-beb4-497d-b09f-b1a62b0593e0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:09.052200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1167/2319] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:18:08 2022] PUT /identity/v3/projects/164df8b44ce844e58f7d168409990608/users/d8eee1ec9ecd4dfbae1864be468894f1/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:18:14.233962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51996001-057d-44e0-a85e-988136617eaf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:14.234527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51996001-057d-44e0-a85e-988136617eaf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:14.234527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51996001-057d-44e0-a85e-988136617eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:14.286881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-51996001-057d-44e0-a85e-988136617eaf None None] MFA Rules not processed for user `3edc8b9046ac4112a838a9c3b39aecf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:14.288608 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-51996001-057d-44e0-a85e-988136617eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:14.449076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1153/2320] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:14.453746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdc73e-beb4-497d-b09f-b1a62b0593e0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:14.575333 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdc73e-beb4-497d-b09f-b1a62b0593e0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:14.586652 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885de4d-a0bd-4797-a18b-e9c14ef51d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:14.587307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885de4d-a0bd-4797-a18b-e9c14ef51d46 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:14.587589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885de4d-a0bd-4797-a18b-e9c14ef51d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:14.587863 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885de4d-a0bd-4797-a18b-e9c14ef51d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:14.588505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885de4d-a0bd-4797-a18b-e9c14ef51d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:14.595364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885de4d-a0bd-4797-a18b-e9c14ef51d46 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:14.651562 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1168/2321] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:18:14 2022] POST /identity/v3/projects => generated 371 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:18:14.653213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51996001-057d-44e0-a85e-988136617eaf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:14.678944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51996001-057d-44e0-a85e-988136617eaf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:14.899912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449ff7f-7d4f-4c6d-8686-5ed6a06ad04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:14.901136 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449ff7f-7d4f-4c6d-8686-5ed6a06ad04f admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:14.902500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449ff7f-7d4f-4c6d-8686-5ed6a06ad04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:14.903132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449ff7f-7d4f-4c6d-8686-5ed6a06ad04f admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:14.904056 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449ff7f-7d4f-4c6d-8686-5ed6a06ad04f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:14.908286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449ff7f-7d4f-4c6d-8686-5ed6a06ad04f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:14.939829 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:14.966043 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1154/2322] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:18:14 2022] POST /identity/v3/users => generated 292 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:18:14.970400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885de4d-a0bd-4797-a18b-e9c14ef51d46 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:14.999754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885de4d-a0bd-4797-a18b-e9c14ef51d46 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:15.009468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194b37b-dec6-4a50-a8aa-b6bbeec55979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:15.010243 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194b37b-dec6-4a50-a8aa-b6bbeec55979 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:15.010630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194b37b-dec6-4a50-a8aa-b6bbeec55979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:15.011619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194b37b-dec6-4a50-a8aa-b6bbeec55979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:15.012528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194b37b-dec6-4a50-a8aa-b6bbeec55979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:15.014506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194b37b-dec6-4a50-a8aa-b6bbeec55979 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:15.028092 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1169/2323] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:18:15.032365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449ff7f-7d4f-4c6d-8686-5ed6a06ad04f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:15.047607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449ff7f-7d4f-4c6d-8686-5ed6a06ad04f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:15.066441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d22ebb-91bc-4a2e-9dc7-c7e27fb9ad10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:15.067501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d22ebb-91bc-4a2e-9dc7-c7e27fb9ad10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:15.067937 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d22ebb-91bc-4a2e-9dc7-c7e27fb9ad10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:15.068364 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d22ebb-91bc-4a2e-9dc7-c7e27fb9ad10 admin admin] PATH_INFO: `/v3/projects/a89d432a742849419bc76f45b2c306c5/users/cb2d34d0ab1c4c38b1f55c13f8af08d6/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:15.096433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d22ebb-91bc-4a2e-9dc7-c7e27fb9ad10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89d432a742849419bc76f45b2c306c5, user_id=cb2d34d0ab1c4c38b1f55c13f8af08d6, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:15.099600 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d22ebb-91bc-4a2e-9dc7-c7e27fb9ad10 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:15.142615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1155/2324] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:18:15 2022] PUT /identity/v3/projects/a89d432a742849419bc76f45b2c306c5/users/cb2d34d0ab1c4c38b1f55c13f8af08d6/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:18:15.146541 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194b37b-dec6-4a50-a8aa-b6bbeec55979 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:15.161573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194b37b-dec6-4a50-a8aa-b6bbeec55979 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:15.226551 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:15.275664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776ded3-5b49-4541-9db7-937834bdbd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:15.276791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776ded3-5b49-4541-9db7-937834bdbd07 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:15.277172 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776ded3-5b49-4541-9db7-937834bdbd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:15.277555 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776ded3-5b49-4541-9db7-937834bdbd07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:15.278418 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776ded3-5b49-4541-9db7-937834bdbd07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:15.280163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776ded3-5b49-4541-9db7-937834bdbd07 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:15.309180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1170/2325] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:18:15.311134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d22ebb-91bc-4a2e-9dc7-c7e27fb9ad10 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:15.332726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d22ebb-91bc-4a2e-9dc7-c7e27fb9ad10 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:15.343105 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a3a9a-c8d3-4813-a6dc-d0cf20cdad97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:15.343898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a3a9a-c8d3-4813-a6dc-d0cf20cdad97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:15.344281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a3a9a-c8d3-4813-a6dc-d0cf20cdad97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:15.344668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a3a9a-c8d3-4813-a6dc-d0cf20cdad97 admin admin] PATH_INFO: `/v3/projects/a89d432a742849419bc76f45b2c306c5/users/cb2d34d0ab1c4c38b1f55c13f8af08d6/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:15.350686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a3a9a-c8d3-4813-a6dc-d0cf20cdad97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89d432a742849419bc76f45b2c306c5, user_id=cb2d34d0ab1c4c38b1f55c13f8af08d6, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:15.355673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a3a9a-c8d3-4813-a6dc-d0cf20cdad97 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:15.393563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1156/2326] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:18:15 2022] PUT /identity/v3/projects/a89d432a742849419bc76f45b2c306c5/users/cb2d34d0ab1c4c38b1f55c13f8af08d6/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:18:16.091521 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c38ec-a3e7-4d71-89a2-96f59a459bc2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:16.092013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c38ec-a3e7-4d71-89a2-96f59a459bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:16.092103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c38ec-a3e7-4d71-89a2-96f59a459bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:16.131497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9b1c38ec-a3e7-4d71-89a2-96f59a459bc2 None None] MFA Rules not processed for user `d8eee1ec9ecd4dfbae1864be468894f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:16.132733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9b1c38ec-a3e7-4d71-89a2-96f59a459bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:16.265068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1171/2327] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:18:16.279918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a3a9a-c8d3-4813-a6dc-d0cf20cdad97 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:16.282400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b20a3a9a-c8d3-4813-a6dc-d0cf20cdad97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:16.390180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a3a9a-c8d3-4813-a6dc-d0cf20cdad97 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:16.402310 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcce89-2953-4e84-8043-6e0bd08b271c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ef6cd64182474787134e3227beb3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:16.403261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcce89-2953-4e84-8043-6e0bd08b271c service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:16.403652 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcce89-2953-4e84-8043-6e0bd08b271c service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:16.404145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcce89-2953-4e84-8043-6e0bd08b271c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:16.513625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcce89-2953-4e84-8043-6e0bd08b271c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:16.515191 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcce89-2953-4e84-8043-6e0bd08b271c service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:16.526482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1157/2328] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:18:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:18:16.725189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c177c-419b-468c-a92a-532564677a38 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:16.725738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c177c-419b-468c-a92a-532564677a38 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:16.726168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c177c-419b-468c-a92a-532564677a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:16.763690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b34c177c-419b-468c-a92a-532564677a38 None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:16.764960 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b34c177c-419b-468c-a92a-532564677a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:16.840656 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1172/2329] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:18:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:16.871949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcce89-2953-4e84-8043-6e0bd08b271c service glance] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:16.873645 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-02dcce89-2953-4e84-8043-6e0bd08b271c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:16.984528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcce89-2953-4e84-8043-6e0bd08b271c service glance] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:16.990912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14282729-4833-41a8-a86e-522111c255d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:16.991794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14282729-4833-41a8-a86e-522111c255d5 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:16.992517 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14282729-4833-41a8-a86e-522111c255d5 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:16.993951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14282729-4833-41a8-a86e-522111c255d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:17.005530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14282729-4833-41a8-a86e-522111c255d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:17.007495 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14282729-4833-41a8-a86e-522111c255d5 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:17.013543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:17.018766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1158/2330] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:18:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:18:23.581015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ca7b5-d040-4847-a04c-d9a4bf90a4bb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:23.581614 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ca7b5-d040-4847-a04c-d9a4bf90a4bb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:23.581614 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ca7b5-d040-4847-a04c-d9a4bf90a4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:23.727579 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-998ca7b5-d040-4847-a04c-d9a4bf90a4bb None None] MFA Rules not processed for user `cb2d34d0ab1c4c38b1f55c13f8af08d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:23.728958 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-998ca7b5-d040-4847-a04c-d9a4bf90a4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:23.869184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:23.905474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1173/2331] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:23.921262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14282729-4833-41a8-a86e-522111c255d5 service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:23.922799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-14282729-4833-41a8-a86e-522111c255d5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:24.028949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14282729-4833-41a8-a86e-522111c255d5 service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:24.041461 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f075a3a-523c-45c4-8ad7-eb5b00878001 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:24.044477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f075a3a-523c-45c4-8ad7-eb5b00878001 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:24.056022 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f075a3a-523c-45c4-8ad7-eb5b00878001 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:24.056417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f075a3a-523c-45c4-8ad7-eb5b00878001 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:24.153382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f075a3a-523c-45c4-8ad7-eb5b00878001 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:24.155176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f075a3a-523c-45c4-8ad7-eb5b00878001 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:24.174241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1159/2332] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:18:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:18:26.194886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae75901-2a12-480d-bafc-a7dea57adb7e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:26.195414 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae75901-2a12-480d-bafc-a7dea57adb7e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:26.196636 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae75901-2a12-480d-bafc-a7dea57adb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:26.246645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-aae75901-2a12-480d-bafc-a7dea57adb7e None None] MFA Rules not processed for user `3edc8b9046ac4112a838a9c3b39aecf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:26.251083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aae75901-2a12-480d-bafc-a7dea57adb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:26.284420 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1174/2333] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:31.735946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe15fd-ddbe-4542-a86d-8fff6aab0468 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:31.736489 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe15fd-ddbe-4542-a86d-8fff6aab0468 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:31.736489 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe15fd-ddbe-4542-a86d-8fff6aab0468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:31.757324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-58fe15fd-ddbe-4542-a86d-8fff6aab0468 None None] MFA Rules not processed for user `b754de037a8148358efb03ffa39786d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:31.758548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-58fe15fd-ddbe-4542-a86d-8fff6aab0468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:31.767090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1160/2334] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:18:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:36.218007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=10 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:41.310598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922ae3e-c839-4313-9782-ae605b507114 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:41.311272 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922ae3e-c839-4313-9782-ae605b507114 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:41.311272 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922ae3e-c839-4313-9782-ae605b507114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:41.333194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a922ae3e-c839-4313-9782-ae605b507114 None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:41.335242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a922ae3e-c839-4313-9782-ae605b507114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:41.343756 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1175/2335] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:18:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:41.358911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe15fd-ddbe-4542-a86d-8fff6aab0468 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:41.377352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe15fd-ddbe-4542-a86d-8fff6aab0468 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:41.385447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee087c9-47d1-451f-b138-a0d37ecd2414 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:41.387650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee087c9-47d1-451f-b138-a0d37ecd2414 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:41.388134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee087c9-47d1-451f-b138-a0d37ecd2414 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:41.388477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee087c9-47d1-451f-b138-a0d37ecd2414 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:41.398016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee087c9-47d1-451f-b138-a0d37ecd2414 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:41.399518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee087c9-47d1-451f-b138-a0d37ecd2414 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:41.410184 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1161/2336] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:18:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:18:51.326774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:51.406149 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=65 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:18:52.259388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828a2db-8ddb-4095-9922-1603572c75f1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:52.260119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828a2db-8ddb-4095-9922-1603572c75f1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:52.261326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828a2db-8ddb-4095-9922-1603572c75f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:52.293947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6828a2db-8ddb-4095-9922-1603572c75f1 None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:18:52.295230 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6828a2db-8ddb-4095-9922-1603572c75f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:18:52.305366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1176/2337] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:18:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:18:52.320341 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee087c9-47d1-451f-b138-a0d37ecd2414 service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:18:52.348456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee087c9-47d1-451f-b138-a0d37ecd2414 service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:18:52.354990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51710a-e3b0-4f72-8ba9-52ffd12f1ed0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:18:52.356206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51710a-e3b0-4f72-8ba9-52ffd12f1ed0 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:18:52.356713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51710a-e3b0-4f72-8ba9-52ffd12f1ed0 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:18:52.357616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51710a-e3b0-4f72-8ba9-52ffd12f1ed0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:18:52.370282 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51710a-e3b0-4f72-8ba9-52ffd12f1ed0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:18:52.372881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51710a-e3b0-4f72-8ba9-52ffd12f1ed0 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:18:52.384312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1162/2338] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:18:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:19:02.298902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:19:02.379691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:19:12.149474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adbf0a-62b7-4191-928f-4ec61c2e13d0 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:12.150109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adbf0a-62b7-4191-928f-4ec61c2e13d0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:12.150109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adbf0a-62b7-4191-928f-4ec61c2e13d0 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:12.151504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1177/2339] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:19:12.157401 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c9f93-0490-4000-8542-8e80478031e8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:12.157908 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c9f93-0490-4000-8542-8e80478031e8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:12.158032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c9f93-0490-4000-8542-8e80478031e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:12.180991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-746c9f93-0490-4000-8542-8e80478031e8 None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:19:12.183494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-746c9f93-0490-4000-8542-8e80478031e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:12.195650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1163/2340] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:19:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:19:12.209076 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adbf0a-62b7-4191-928f-4ec61c2e13d0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:19:12.222172 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-53adbf0a-62b7-4191-928f-4ec61c2e13d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:12.237799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adbf0a-62b7-4191-928f-4ec61c2e13d0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:19:12.241498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a2e41-200e-4b59-90e2-a8aca0715504 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ef6cd64182474787134e3227beb3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:19:12.242110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2e41-200e-4b59-90e2-a8aca0715504 service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:12.242392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2e41-200e-4b59-90e2-a8aca0715504 service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:12.242663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2e41-200e-4b59-90e2-a8aca0715504 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:12.254083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a2e41-200e-4b59-90e2-a8aca0715504 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:19:12.255560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a2e41-200e-4b59-90e2-a8aca0715504 service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:19:12.268544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1178/2341] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:19:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:19:12.339789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679825c-fe24-487c-a89e-627e03e27e0e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:12.340079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679825c-fe24-487c-a89e-627e03e27e0e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:12.340440 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679825c-fe24-487c-a89e-627e03e27e0e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:12.341894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1164/2342] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:19:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:19:12.346858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a554efa-ceb6-40da-aa62-4048e7e87eff None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:12.347297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a554efa-ceb6-40da-aa62-4048e7e87eff None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:12.347576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a554efa-ceb6-40da-aa62-4048e7e87eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:12.371156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0a554efa-ceb6-40da-aa62-4048e7e87eff None None] MFA Rules not processed for user `e12deabe077b4095bfa7e2e8fac61d14`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:19:12.372416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0a554efa-ceb6-40da-aa62-4048e7e87eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:12.431235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1179/2343] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:19:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:19:12.443976 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c679825c-fe24-487c-a89e-627e03e27e0e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:19:12.445702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c679825c-fe24-487c-a89e-627e03e27e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:12.459692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c679825c-fe24-487c-a89e-627e03e27e0e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:19:12.463336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16660dbb-d8cc-4dc4-9fe8-275284c37841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:19:12.464009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16660dbb-d8cc-4dc4-9fe8-275284c37841 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:12.464370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16660dbb-d8cc-4dc4-9fe8-275284c37841 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:12.464703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16660dbb-d8cc-4dc4-9fe8-275284c37841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:12.471790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16660dbb-d8cc-4dc4-9fe8-275284c37841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:19:12.473292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16660dbb-d8cc-4dc4-9fe8-275284c37841 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:19:12.481458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1165/2344] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:19:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:19:12.577729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc1382-937b-44f7-99ba-19ffa03e4703 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:12.578051 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc1382-937b-44f7-99ba-19ffa03e4703 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:12.579654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc1382-937b-44f7-99ba-19ffa03e4703 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:12.583130 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1180/2345] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:19:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:19:12.600563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4b960-5f19-4e2c-8310-19f78981820c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:12.600850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4b960-5f19-4e2c-8310-19f78981820c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:12.601130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4b960-5f19-4e2c-8310-19f78981820c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:12.622706 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3fb4b960-5f19-4e2c-8310-19f78981820c None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:19:12.623977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3fb4b960-5f19-4e2c-8310-19f78981820c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:12.633987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1166/2346] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:19:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:19:12.648052 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc1382-937b-44f7-99ba-19ffa03e4703 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:19:12.661077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc1382-937b-44f7-99ba-19ffa03e4703 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:19:12.665070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79db81-68e9-427c-8679-3cd4e2f67948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:19:12.665794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79db81-68e9-427c-8679-3cd4e2f67948 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:12.665942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79db81-68e9-427c-8679-3cd4e2f67948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:12.666214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79db81-68e9-427c-8679-3cd4e2f67948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:12.677310 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79db81-68e9-427c-8679-3cd4e2f67948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:19:12.678828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79db81-68e9-427c-8679-3cd4e2f67948 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:19:12.693521 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1181/2347] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:19:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:19:14.102275 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da251660-0888-48d3-a48e-58750b1208ef None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:14.102762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da251660-0888-48d3-a48e-58750b1208ef None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:14.102858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da251660-0888-48d3-a48e-58750b1208ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:14.122946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-da251660-0888-48d3-a48e-58750b1208ef None None] MFA Rules not processed for user `dcb88983a8e1439588644b71170d7362`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:19:14.124189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-da251660-0888-48d3-a48e-58750b1208ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:14.183489 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1167/2348] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:19:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:19:14.680891 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639c011-ee28-49c0-bc61-4a7811d98928 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:14.681397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639c011-ee28-49c0-bc61-4a7811d98928 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:14.681485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639c011-ee28-49c0-bc61-4a7811d98928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:14.704024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a639c011-ee28-49c0-bc61-4a7811d98928 None None] MFA Rules not processed for user `41ec6426ddf640389083cc61e948d31f`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:19:14.705497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a639c011-ee28-49c0-bc61-4a7811d98928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:14.799674 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1182/2349] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:19:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:19:19.003688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da251660-0888-48d3-a48e-58750b1208ef None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:19:19.016379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da251660-0888-48d3-a48e-58750b1208ef None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:19:19.031141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadc188-19db-4cf9-84f7-aa7f30e4764a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:19:19.034054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadc188-19db-4cf9-84f7-aa7f30e4764a service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:19.035018 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadc188-19db-4cf9-84f7-aa7f30e4764a service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:19.035843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadc188-19db-4cf9-84f7-aa7f30e4764a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:19.063897 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dadc188-19db-4cf9-84f7-aa7f30e4764a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:19:19.066243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dadc188-19db-4cf9-84f7-aa7f30e4764a service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:19:19.099602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1168/2350] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:19:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:19:21.152280 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639c011-ee28-49c0-bc61-4a7811d98928 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:19:21.165764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639c011-ee28-49c0-bc61-4a7811d98928 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:19:21.178445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb58785-f5ac-4d37-b01c-9548593ed444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:19:21.182370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb58785-f5ac-4d37-b01c-9548593ed444 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:21.183369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb58785-f5ac-4d37-b01c-9548593ed444 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:21.183744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb58785-f5ac-4d37-b01c-9548593ed444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:21.211071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb58785-f5ac-4d37-b01c-9548593ed444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:19:21.212678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb58785-f5ac-4d37-b01c-9548593ed444 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:19:21.234706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1183/2351] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:19:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:19:25.250571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ebbe9-5c5a-4efc-b914-58ffbc8d334a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:25.251227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ebbe9-5c5a-4efc-b914-58ffbc8d334a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:25.251227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ebbe9-5c5a-4efc-b914-58ffbc8d334a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:25.270921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-333ebbe9-5c5a-4efc-b914-58ffbc8d334a None None] MFA Rules not processed for user `21700a3c2ff24fc6a3bb5db8bf283f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:19:25.272421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-333ebbe9-5c5a-4efc-b914-58ffbc8d334a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:25.337086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1169/2352] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:19:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:19:26.364877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bcfff-8dca-46cb-8863-f94d37a33c92 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:26.365340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bcfff-8dca-46cb-8863-f94d37a33c92 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:26.365423 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bcfff-8dca-46cb-8863-f94d37a33c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:26.400150 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b08bcfff-8dca-46cb-8863-f94d37a33c92 None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:19:26.438196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b08bcfff-8dca-46cb-8863-f94d37a33c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:26.454659 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1184/2353] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:19:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:19:26.476723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ebbe9-5c5a-4efc-b914-58ffbc8d334a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:19:26.518206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ebbe9-5c5a-4efc-b914-58ffbc8d334a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:19:26.533299 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82704935-6036-4351-b0ea-439a4f93c2bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:19:26.534023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82704935-6036-4351-b0ea-439a4f93c2bf service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:26.534374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82704935-6036-4351-b0ea-439a4f93c2bf service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:26.534713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82704935-6036-4351-b0ea-439a4f93c2bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:26.550216 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82704935-6036-4351-b0ea-439a4f93c2bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:19:26.552190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82704935-6036-4351-b0ea-439a4f93c2bf service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:19:26.562023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1170/2354] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:19:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:19:36.383423 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=69 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:19:36.557604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=74 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:19:38.748493 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08bcfff-8dca-46cb-8863-f94d37a33c92 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:19:38.761030 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08bcfff-8dca-46cb-8863-f94d37a33c92 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:19:38.764817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053a2f6-147a-416f-ba13-71b1f635517b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:19:38.765418 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053a2f6-147a-416f-ba13-71b1f635517b service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:38.765691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053a2f6-147a-416f-ba13-71b1f635517b service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:38.765959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053a2f6-147a-416f-ba13-71b1f635517b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:38.767716 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0053a2f6-147a-416f-ba13-71b1f635517b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:38.785489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053a2f6-147a-416f-ba13-71b1f635517b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:19:38.787378 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053a2f6-147a-416f-ba13-71b1f635517b service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:19:38.803644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1185/2355] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:19:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:19:41.212671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82704935-6036-4351-b0ea-439a4f93c2bf service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:19:41.221929 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82704935-6036-4351-b0ea-439a4f93c2bf service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:19:41.225473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bec4de-312b-45c5-9b3a-1ee4d31df2e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:19:41.226054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bec4de-312b-45c5-9b3a-1ee4d31df2e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:41.226330 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bec4de-312b-45c5-9b3a-1ee4d31df2e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:41.226595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bec4de-312b-45c5-9b3a-1ee4d31df2e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:41.229067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f6bec4de-312b-45c5-9b3a-1ee4d31df2e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:41.241812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bec4de-312b-45c5-9b3a-1ee4d31df2e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:19:41.243258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bec4de-312b-45c5-9b3a-1ee4d31df2e2 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:19:41.254719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1171/2356] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:19:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:19:48.796809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:19:51.250164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:19:53.311592 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1ff78-078f-4adb-8fcf-c93cf24786cc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:53.312042 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1ff78-078f-4adb-8fcf-c93cf24786cc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:53.312156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1ff78-078f-4adb-8fcf-c93cf24786cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:53.332172 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c0a1ff78-078f-4adb-8fcf-c93cf24786cc None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:19:53.333376 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c0a1ff78-078f-4adb-8fcf-c93cf24786cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:53.342642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1186/2357] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 9 13:19:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:19:53.356905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bec4de-312b-45c5-9b3a-1ee4d31df2e2 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:19:53.368092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bec4de-312b-45c5-9b3a-1ee4d31df2e2 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:19:53.374320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315357a4-330d-489c-bb21-ed1f8a3ec3be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:19:53.374946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315357a4-330d-489c-bb21-ed1f8a3ec3be service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:53.375265 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315357a4-330d-489c-bb21-ed1f8a3ec3be service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:53.375539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315357a4-330d-489c-bb21-ed1f8a3ec3be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:53.383907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315357a4-330d-489c-bb21-ed1f8a3ec3be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:19:53.386490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315357a4-330d-489c-bb21-ed1f8a3ec3be service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:19:53.394885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1172/2358] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:19:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:19:58.106572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1ff78-078f-4adb-8fcf-c93cf24786cc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:19:58.108000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c0a1ff78-078f-4adb-8fcf-c93cf24786cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:58.163256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1ff78-078f-4adb-8fcf-c93cf24786cc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:19:58.167023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe645359-2cf5-4770-bf0e-146cd158c773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:19:58.167653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe645359-2cf5-4770-bf0e-146cd158c773 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:19:58.167934 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe645359-2cf5-4770-bf0e-146cd158c773 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:19:58.168205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe645359-2cf5-4770-bf0e-146cd158c773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:19:58.169918 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fe645359-2cf5-4770-bf0e-146cd158c773 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:19:58.230192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe645359-2cf5-4770-bf0e-146cd158c773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:19:58.231679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe645359-2cf5-4770-bf0e-146cd158c773 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:19:58.240795 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1187/2359] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:19:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:20:03.391593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:20:08.236491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=51 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:20:08.772896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bde6d-32c1-4b32-90ef-564fac0cc25a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:20:08.773409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bde6d-32c1-4b32-90ef-564fac0cc25a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:20:08.773625 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bde6d-32c1-4b32-90ef-564fac0cc25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:20:08.794042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fb6bde6d-32c1-4b32-90ef-564fac0cc25a None None] MFA Rules not processed for user `c31bb28e38864eff99fdd6938b1b89a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:20:08.795294 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fb6bde6d-32c1-4b32-90ef-564fac0cc25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:20:08.802145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1173/2360] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:20:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:20:09.108112 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe645359-2cf5-4770-bf0e-146cd158c773 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:20:09.117311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe645359-2cf5-4770-bf0e-146cd158c773 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:20:09.121339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cc3c2-998c-4e4e-a5c0-c817dacc73b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:20:09.121936 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cc3c2-998c-4e4e-a5c0-c817dacc73b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:20:09.122235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cc3c2-998c-4e4e-a5c0-c817dacc73b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:20:09.122489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cc3c2-998c-4e4e-a5c0-c817dacc73b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:20:09.124239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4d0cc3c2-998c-4e4e-a5c0-c817dacc73b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:20:09.133434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cc3c2-998c-4e4e-a5c0-c817dacc73b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:20:09.135557 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cc3c2-998c-4e4e-a5c0-c817dacc73b0 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:20:09.143319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1188/2361] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:20:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:20:09.457270 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bde6d-32c1-4b32-90ef-564fac0cc25a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:20:09.458813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fb6bde6d-32c1-4b32-90ef-564fac0cc25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:20:09.539763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bde6d-32c1-4b32-90ef-564fac0cc25a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:20:09.545068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c29cc-efc8-4201-b26f-e491ad2082e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:20:09.545685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c29cc-efc8-4201-b26f-e491ad2082e3 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:20:09.545972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c29cc-efc8-4201-b26f-e491ad2082e3 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:20:09.546256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c29cc-efc8-4201-b26f-e491ad2082e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:20:09.548184 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1a5c29cc-efc8-4201-b26f-e491ad2082e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:20:09.560817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c29cc-efc8-4201-b26f-e491ad2082e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:20:09.562410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c29cc-efc8-4201-b26f-e491ad2082e3 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:20:09.573014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1174/2362] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:20:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:20:12.851680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cc3c2-998c-4e4e-a5c0-c817dacc73b0 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:20:12.853180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4d0cc3c2-998c-4e4e-a5c0-c817dacc73b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:20:12.869886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cc3c2-998c-4e4e-a5c0-c817dacc73b0 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:20:12.875884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e14e3-2eb5-4715-a9b5-4fdb698fc2bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:20:12.876503 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e14e3-2eb5-4715-a9b5-4fdb698fc2bd service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:20:12.876784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e14e3-2eb5-4715-a9b5-4fdb698fc2bd service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:20:12.877062 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e14e3-2eb5-4715-a9b5-4fdb698fc2bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:20:12.878808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-897e14e3-2eb5-4715-a9b5-4fdb698fc2bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:20:12.892789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e14e3-2eb5-4715-a9b5-4fdb698fc2bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:20:12.894995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e14e3-2eb5-4715-a9b5-4fdb698fc2bd service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:20:12.907170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1189/2363] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:20:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:20:14.321663 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c29cc-efc8-4201-b26f-e491ad2082e3 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:20:14.333110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c29cc-efc8-4201-b26f-e491ad2082e3 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:20:14.337075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d229f6-1e3d-4535-aae7-56648280f1ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:20:14.338452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d229f6-1e3d-4535-aae7-56648280f1ea service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:20:14.338943 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d229f6-1e3d-4535-aae7-56648280f1ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:20:14.339388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d229f6-1e3d-4535-aae7-56648280f1ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:20:14.341160 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-86d229f6-1e3d-4535-aae7-56648280f1ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:20:14.351231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d229f6-1e3d-4535-aae7-56648280f1ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:20:14.352813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d229f6-1e3d-4535-aae7-56648280f1ea service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:20:14.362475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1175/2364] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:20:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:20:22.901503 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=12 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:20:24.358594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=41 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:20:38.318455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e14e3-2eb5-4715-a9b5-4fdb698fc2bd service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:20:38.320944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-897e14e3-2eb5-4715-a9b5-4fdb698fc2bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:20:38.399203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e14e3-2eb5-4715-a9b5-4fdb698fc2bd service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:20:38.404682 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:20:38.405371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:20:38.405650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:20:38.405920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] PATH_INFO: `/v3/users/41ec6426ddf640389083cc61e948d31f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:20:38.407045 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ec6426ddf640389083cc61e948d31f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:20:38.410800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:20:38.457695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ec6426ddf640389083cc61e948d31f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:20:38.464776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ec6426ddf640389083cc61e948d31f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:20:38.465398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ec6426ddf640389083cc61e948d31f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:20:38.477263 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ec6426ddf640389083cc61e948d31f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:20:38.484560 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ec6426ddf640389083cc61e948d31f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:20:38.492886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1190/2365] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:20:38 2022] DELETE /identity/v3/users/41ec6426ddf640389083cc61e948d31f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:20:38.693907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d229f6-1e3d-4535-aae7-56648280f1ea service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:20:38.708917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d229f6-1e3d-4535-aae7-56648280f1ea service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:20:38.772132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:20:38.773013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:20:38.773308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:20:38.773583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] PATH_INFO: `/v3/projects/345a9eb6369c43a681649f1bb55ce60c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:20:38.774757 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=345a9eb6369c43a681649f1bb55ce60c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:20:38.776717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:20:38.828472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345a9eb6369c43a681649f1bb55ce60c', 'request_id': 'req-4b43ee80-6efb-4b39-9779-93da0f9fae39'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:20:38.834883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345a9eb6369c43a681649f1bb55ce60c', 'request_id': 'req-4b43ee80-6efb-4b39-9779-93da0f9fae39'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:20:38.841499 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345a9eb6369c43a681649f1bb55ce60c', 'request_id': 'req-4b43ee80-6efb-4b39-9779-93da0f9fae39'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:20:38.856059 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] The token cache is being invalidate because project 345a9eb6369c43a681649f1bb55ce60c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:20:38.856345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:20:38.856983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:20:38.859502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1176/2366] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:20:38 2022] DELETE /identity/v3/projects/345a9eb6369c43a681649f1bb55ce60c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:20:43.045506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:20:43.047031 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:20:43.107878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a54130-d3b8-4ebd-ab07-64a48e534c80 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:20:43.141154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:20:43.159513 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e161828-52eb-4776-86d0-2518a2d320fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:20:43.160144 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e161828-52eb-4776-86d0-2518a2d320fc service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:20:43.160425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e161828-52eb-4776-86d0-2518a2d320fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:20:43.160695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e161828-52eb-4776-86d0-2518a2d320fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:20:43.162465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2e161828-52eb-4776-86d0-2518a2d320fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:20:43.222698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e161828-52eb-4776-86d0-2518a2d320fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:20:43.224177 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e161828-52eb-4776-86d0-2518a2d320fc service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:20:43.234223 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1191/2367] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:20:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:20:48.852508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:20:53.228897 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=32 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:20:54.899242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:20:54.900575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:20:54.974428 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43ee80-6efb-4b39-9779-93da0f9fae39 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:20:54.979928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadba389-cc65-438c-a4cc-67992ffcc1ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:20:54.980538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadba389-cc65-438c-a4cc-67992ffcc1ea service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:20:54.980818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadba389-cc65-438c-a4cc-67992ffcc1ea service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:20:54.981094 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadba389-cc65-438c-a4cc-67992ffcc1ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:20:54.983418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fadba389-cc65-438c-a4cc-67992ffcc1ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:20:55.041528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadba389-cc65-438c-a4cc-67992ffcc1ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:20:55.043151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadba389-cc65-438c-a4cc-67992ffcc1ea service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:20:55.052251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1177/2368] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:20:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:20:57.182542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc9768-1690-4ead-906b-f10d0eef6593 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:20:57.183021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc9768-1690-4ead-906b-f10d0eef6593 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:20:57.183142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc9768-1690-4ead-906b-f10d0eef6593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:20:57.202672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d3bc9768-1690-4ead-906b-f10d0eef6593 None None] MFA Rules not processed for user `ca9eb6b9dc6e409a954c7993ebe5a5c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:20:57.203927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d3bc9768-1690-4ead-906b-f10d0eef6593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:20:57.258810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1192/2369] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:20:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:05.048049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=56 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:06.027181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadba389-cc65-438c-a4cc-67992ffcc1ea service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:06.028600 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fadba389-cc65-438c-a4cc-67992ffcc1ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:06.114055 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadba389-cc65-438c-a4cc-67992ffcc1ea service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:06.120343 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6ba5f-78d6-4ce4-9f71-c6ec5f36d9e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:06.121649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6ba5f-78d6-4ce4-9f71-c6ec5f36d9e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:06.122145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6ba5f-78d6-4ce4-9f71-c6ec5f36d9e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:06.123243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6ba5f-78d6-4ce4-9f71-c6ec5f36d9e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:06.126101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-79c6ba5f-78d6-4ce4-9f71-c6ec5f36d9e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:06.201506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6ba5f-78d6-4ce4-9f71-c6ec5f36d9e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:06.202874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6ba5f-78d6-4ce4-9f71-c6ec5f36d9e4 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:06.212867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1178/2370] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:07.252145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=25 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:09.733826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc9768-1690-4ead-906b-f10d0eef6593 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:09.736317 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d3bc9768-1690-4ead-906b-f10d0eef6593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:09.782209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc9768-1690-4ead-906b-f10d0eef6593 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:09.810002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa4410-98e9-434c-bfc5-7b72a5c06752 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:09.812831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa4410-98e9-434c-bfc5-7b72a5c06752 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:09.813298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa4410-98e9-434c-bfc5-7b72a5c06752 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:09.816085 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa4410-98e9-434c-bfc5-7b72a5c06752 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:09.820580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e1aa4410-98e9-434c-bfc5-7b72a5c06752 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:09.870157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa4410-98e9-434c-bfc5-7b72a5c06752 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:09.872507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa4410-98e9-434c-bfc5-7b72a5c06752 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:09.907881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1193/2371] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:21:10.421471 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6ba5f-78d6-4ce4-9f71-c6ec5f36d9e4 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:10.424113 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-79c6ba5f-78d6-4ce4-9f71-c6ec5f36d9e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:10.446619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6ba5f-78d6-4ce4-9f71-c6ec5f36d9e4 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:10.453176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef27a53-360d-43f9-bcb1-b60ca413c57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:10.453980 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef27a53-360d-43f9-bcb1-b60ca413c57e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:10.454364 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef27a53-360d-43f9-bcb1-b60ca413c57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:10.455328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef27a53-360d-43f9-bcb1-b60ca413c57e admin admin] PATH_INFO: `/v3/users/21700a3c2ff24fc6a3bb5db8bf283f5c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:10.456599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef27a53-360d-43f9-bcb1-b60ca413c57e admin admin] RBAC: Authorizing `identity:delete_user(user_id=21700a3c2ff24fc6a3bb5db8bf283f5c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:10.458808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef27a53-360d-43f9-bcb1-b60ca413c57e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:10.533243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0ef27a53-360d-43f9-bcb1-b60ca413c57e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21700a3c2ff24fc6a3bb5db8bf283f5c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:10.545553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0ef27a53-360d-43f9-bcb1-b60ca413c57e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21700a3c2ff24fc6a3bb5db8bf283f5c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:10.560616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0ef27a53-360d-43f9-bcb1-b60ca413c57e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21700a3c2ff24fc6a3bb5db8bf283f5c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:10.561541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0ef27a53-360d-43f9-bcb1-b60ca413c57e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21700a3c2ff24fc6a3bb5db8bf283f5c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:10.572784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0ef27a53-360d-43f9-bcb1-b60ca413c57e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21700a3c2ff24fc6a3bb5db8bf283f5c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:10.583147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1179/2372] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:10 2022] DELETE /identity/v3/users/21700a3c2ff24fc6a3bb5db8bf283f5c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:10.778354 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa4410-98e9-434c-bfc5-7b72a5c06752 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:10.799970 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa4410-98e9-434c-bfc5-7b72a5c06752 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:10.856964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712784c9-05f9-421c-9dab-900cf1d51657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:10.858513 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712784c9-05f9-421c-9dab-900cf1d51657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:10.858802 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712784c9-05f9-421c-9dab-900cf1d51657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:10.859099 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712784c9-05f9-421c-9dab-900cf1d51657 admin admin] PATH_INFO: `/v3/projects/1eedefa6b656409f902d158e77d94aa0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:10.860242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712784c9-05f9-421c-9dab-900cf1d51657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eedefa6b656409f902d158e77d94aa0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:10.862142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712784c9-05f9-421c-9dab-900cf1d51657 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:10.910433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-712784c9-05f9-421c-9dab-900cf1d51657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eedefa6b656409f902d158e77d94aa0', 'request_id': 'req-712784c9-05f9-421c-9dab-900cf1d51657'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:10.918336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-712784c9-05f9-421c-9dab-900cf1d51657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eedefa6b656409f902d158e77d94aa0', 'request_id': 'req-712784c9-05f9-421c-9dab-900cf1d51657'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:10.925098 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-712784c9-05f9-421c-9dab-900cf1d51657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eedefa6b656409f902d158e77d94aa0', 'request_id': 'req-712784c9-05f9-421c-9dab-900cf1d51657'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:10.936941 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-712784c9-05f9-421c-9dab-900cf1d51657 admin admin] The token cache is being invalidate because project 1eedefa6b656409f902d158e77d94aa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:10.937227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-712784c9-05f9-421c-9dab-900cf1d51657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:10.938187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-712784c9-05f9-421c-9dab-900cf1d51657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:10.939732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1194/2373] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:10 2022] DELETE /identity/v3/projects/1eedefa6b656409f902d158e77d94aa0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:10.945014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39e983-6ca9-48e9-b64c-a0733edfcfe7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:10.945302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39e983-6ca9-48e9-b64c-a0733edfcfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:10.945581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39e983-6ca9-48e9-b64c-a0733edfcfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:10.966155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5c39e983-6ca9-48e9-b64c-a0733edfcfe7 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:10.967987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5c39e983-6ca9-48e9-b64c-a0733edfcfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:10.986293 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:11.033093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1180/2374] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:11.040769 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efa2d7-8b85-4db0-a71d-8f7c3d01ac88 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:11.041329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efa2d7-8b85-4db0-a71d-8f7c3d01ac88 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:11.041603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efa2d7-8b85-4db0-a71d-8f7c3d01ac88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:11.063703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e3efa2d7-8b85-4db0-a71d-8f7c3d01ac88 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:11.065161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e3efa2d7-8b85-4db0-a71d-8f7c3d01ac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:11.077423 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1195/2375] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:11.078453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39e983-6ca9-48e9-b64c-a0733edfcfe7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:11.088874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39e983-6ca9-48e9-b64c-a0733edfcfe7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:11.094175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc77a58-5c0c-47b8-b70d-ba2a6126ad1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:11.094800 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc77a58-5c0c-47b8-b70d-ba2a6126ad1c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:11.095129 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc77a58-5c0c-47b8-b70d-ba2a6126ad1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:11.095830 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc77a58-5c0c-47b8-b70d-ba2a6126ad1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:11.096058 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc77a58-5c0c-47b8-b70d-ba2a6126ad1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:21:11.096510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc77a58-5c0c-47b8-b70d-ba2a6126ad1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:11.098156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc77a58-5c0c-47b8-b70d-ba2a6126ad1c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:11.114834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1181/2376] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:21:11.119397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efa2d7-8b85-4db0-a71d-8f7c3d01ac88 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:11.129933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efa2d7-8b85-4db0-a71d-8f7c3d01ac88 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:11.136142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decfca7-73e7-4796-ae88-c5197c5740a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:11.136545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decfca7-73e7-4796-ae88-c5197c5740a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:11.137005 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decfca7-73e7-4796-ae88-c5197c5740a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:11.137261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decfca7-73e7-4796-ae88-c5197c5740a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:11.138341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7decfca7-73e7-4796-ae88-c5197c5740a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:11.140338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7decfca7-73e7-4796-ae88-c5197c5740a0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:11.183333 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1196/2377] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:11 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:11.184962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc77a58-5c0c-47b8-b70d-ba2a6126ad1c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:11.198100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc77a58-5c0c-47b8-b70d-ba2a6126ad1c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:11.284336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e489f0-7dec-475e-bbc5-75e95ae9da7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:11.285084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e489f0-7dec-475e-bbc5-75e95ae9da7e admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:11.285471 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e489f0-7dec-475e-bbc5-75e95ae9da7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:11.285806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e489f0-7dec-475e-bbc5-75e95ae9da7e admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:11.286518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e489f0-7dec-475e-bbc5-75e95ae9da7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:11.289218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e489f0-7dec-475e-bbc5-75e95ae9da7e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:11.327706 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1182/2378] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:11 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:11.332062 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decfca7-73e7-4796-ae88-c5197c5740a0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:11.345006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decfca7-73e7-4796-ae88-c5197c5740a0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:11.350063 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243e040-1586-4d2a-ad95-cc5011ae3c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:11.350721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243e040-1586-4d2a-ad95-cc5011ae3c9b admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:11.351011 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243e040-1586-4d2a-ad95-cc5011ae3c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:11.351313 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243e040-1586-4d2a-ad95-cc5011ae3c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:11.352350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243e040-1586-4d2a-ad95-cc5011ae3c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:11.356665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243e040-1586-4d2a-ad95-cc5011ae3c9b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:11.368373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1197/2379] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:21:11.373848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e489f0-7dec-475e-bbc5-75e95ae9da7e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:11.387434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e489f0-7dec-475e-bbc5-75e95ae9da7e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:11.392843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ca8de-1b3e-431a-91ee-7177bbeb57fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:11.393451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ca8de-1b3e-431a-91ee-7177bbeb57fa admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:11.393844 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ca8de-1b3e-431a-91ee-7177bbeb57fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:11.394119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ca8de-1b3e-431a-91ee-7177bbeb57fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:11.395015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ca8de-1b3e-431a-91ee-7177bbeb57fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:11.396389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ca8de-1b3e-431a-91ee-7177bbeb57fa admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:11.411182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1183/2380] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:11.414006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243e040-1586-4d2a-ad95-cc5011ae3c9b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:11.426701 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243e040-1586-4d2a-ad95-cc5011ae3c9b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:11.432993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ff853-5725-4183-b6f9-616bc60ab3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:11.433962 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ff853-5725-4183-b6f9-616bc60ab3a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:11.434245 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ff853-5725-4183-b6f9-616bc60ab3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:11.437496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ff853-5725-4183-b6f9-616bc60ab3a7 admin admin] PATH_INFO: `/v3/projects/b348a934b7564433ae70811baedc88d1/users/bdd5a23501b748c3b43f17e04684057a/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:11.455685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ff853-5725-4183-b6f9-616bc60ab3a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b348a934b7564433ae70811baedc88d1, user_id=bdd5a23501b748c3b43f17e04684057a, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:11.458009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ff853-5725-4183-b6f9-616bc60ab3a7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:11.479627 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1198/2381] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:11 2022] PUT /identity/v3/projects/b348a934b7564433ae70811baedc88d1/users/bdd5a23501b748c3b43f17e04684057a/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:21:11.484110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fc0f3-2b04-404e-ac7d-2aae6e9caf39 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:11.484716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fc0f3-2b04-404e-ac7d-2aae6e9caf39 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:11.484716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fc0f3-2b04-404e-ac7d-2aae6e9caf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:11.517136 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bb5fc0f3-2b04-404e-ac7d-2aae6e9caf39 None None] MFA Rules not processed for user `bdd5a23501b748c3b43f17e04684057a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:11.518393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bb5fc0f3-2b04-404e-ac7d-2aae6e9caf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:11.598748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1184/2382] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:11.603311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ff853-5725-4183-b6f9-616bc60ab3a7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:11.618997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:11.672891 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ff853-5725-4183-b6f9-616bc60ab3a7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:11.678276 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cad17d-75bb-4473-af60-dbc24bb849a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:11.678904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cad17d-75bb-4473-af60-dbc24bb849a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:11.679205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cad17d-75bb-4473-af60-dbc24bb849a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:11.679480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cad17d-75bb-4473-af60-dbc24bb849a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:11.680116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cad17d-75bb-4473-af60-dbc24bb849a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:11.682026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cad17d-75bb-4473-af60-dbc24bb849a3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:11.713324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1199/2383] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:11 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:11.714847 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5fc0f3-2b04-404e-ac7d-2aae6e9caf39 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:11.727478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5fc0f3-2b04-404e-ac7d-2aae6e9caf39 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:11.745835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:11.806715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256e88c-74f9-4239-8724-92d0db6869d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:11.806715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256e88c-74f9-4239-8724-92d0db6869d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:11.806715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256e88c-74f9-4239-8724-92d0db6869d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:11.806715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256e88c-74f9-4239-8724-92d0db6869d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:11.812293 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256e88c-74f9-4239-8724-92d0db6869d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:11.812293 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256e88c-74f9-4239-8724-92d0db6869d8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:11.842001 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1185/2384] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:11 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:11.844739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cad17d-75bb-4473-af60-dbc24bb849a3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:11.856465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cad17d-75bb-4473-af60-dbc24bb849a3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:11.863377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca86810-899d-4c40-9f6a-9b7f89137713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:11.864744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca86810-899d-4c40-9f6a-9b7f89137713 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:11.865244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca86810-899d-4c40-9f6a-9b7f89137713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:11.865715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca86810-899d-4c40-9f6a-9b7f89137713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:11.866611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca86810-899d-4c40-9f6a-9b7f89137713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:11.868662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca86810-899d-4c40-9f6a-9b7f89137713 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:11.882305 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1200/2385] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:11.886266 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256e88c-74f9-4239-8724-92d0db6869d8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:11.898583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256e88c-74f9-4239-8724-92d0db6869d8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:11.906637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fa96c-3ab6-4c01-a146-327fcb6d127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:11.907359 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fa96c-3ab6-4c01-a146-327fcb6d127a admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:11.907646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fa96c-3ab6-4c01-a146-327fcb6d127a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:11.908041 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fa96c-3ab6-4c01-a146-327fcb6d127a admin admin] PATH_INFO: `/v3/projects/aedc3c92dd534d26be3eea7be6f3ae72/users/1aa002f9f49544b1b57eef77d79c61e7/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:11.930020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04fa96c-3ab6-4c01-a146-327fcb6d127a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aedc3c92dd534d26be3eea7be6f3ae72, user_id=1aa002f9f49544b1b57eef77d79c61e7, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:11.933105 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04fa96c-3ab6-4c01-a146-327fcb6d127a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:11.952771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1186/2386] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:11 2022] PUT /identity/v3/projects/aedc3c92dd534d26be3eea7be6f3ae72/users/1aa002f9f49544b1b57eef77d79c61e7/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:11.957522 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca86810-899d-4c40-9f6a-9b7f89137713 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:11.968619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca86810-899d-4c40-9f6a-9b7f89137713 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:12.033135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005efdc-67a8-43a7-8e91-62b7b40bbfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:12.033751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005efdc-67a8-43a7-8e91-62b7b40bbfaf admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:12.034030 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005efdc-67a8-43a7-8e91-62b7b40bbfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:12.034296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005efdc-67a8-43a7-8e91-62b7b40bbfaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:12.034935 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005efdc-67a8-43a7-8e91-62b7b40bbfaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:12.036595 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005efdc-67a8-43a7-8e91-62b7b40bbfaf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:12.050980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1201/2387] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:21:12.052614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fa96c-3ab6-4c01-a146-327fcb6d127a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:12.065216 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fa96c-3ab6-4c01-a146-327fcb6d127a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:12.071973 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db553283-4c92-4179-a1b4-a14d54e2e27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:12.072648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db553283-4c92-4179-a1b4-a14d54e2e27d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:12.072930 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db553283-4c92-4179-a1b4-a14d54e2e27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:12.073207 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db553283-4c92-4179-a1b4-a14d54e2e27d admin admin] PATH_INFO: `/v3/projects/aedc3c92dd534d26be3eea7be6f3ae72/users/1aa002f9f49544b1b57eef77d79c61e7/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:12.082857 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db553283-4c92-4179-a1b4-a14d54e2e27d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aedc3c92dd534d26be3eea7be6f3ae72, user_id=1aa002f9f49544b1b57eef77d79c61e7, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:12.086138 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db553283-4c92-4179-a1b4-a14d54e2e27d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:12.104452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1187/2388] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:12 2022] PUT /identity/v3/projects/aedc3c92dd534d26be3eea7be6f3ae72/users/1aa002f9f49544b1b57eef77d79c61e7/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:21:12.114578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3e8d6-b50b-49e5-a985-8c7b8ceef625 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:12.114858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3e8d6-b50b-49e5-a985-8c7b8ceef625 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:12.115230 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3e8d6-b50b-49e5-a985-8c7b8ceef625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:12.146713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-55e3e8d6-b50b-49e5-a985-8c7b8ceef625 None None] MFA Rules not processed for user `1aa002f9f49544b1b57eef77d79c61e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:12.148062 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-55e3e8d6-b50b-49e5-a985-8c7b8ceef625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:12.233080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1202/2389] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:12.255679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db553283-4c92-4179-a1b4-a14d54e2e27d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:12.257569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-db553283-4c92-4179-a1b4-a14d54e2e27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:12.337582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db553283-4c92-4179-a1b4-a14d54e2e27d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:12.342417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7832885-4cf6-4fcd-8c4d-f4be32a04300 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:12.343384 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7832885-4cf6-4fcd-8c4d-f4be32a04300 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:12.343900 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7832885-4cf6-4fcd-8c4d-f4be32a04300 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:12.344588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7832885-4cf6-4fcd-8c4d-f4be32a04300 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:12.354528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7832885-4cf6-4fcd-8c4d-f4be32a04300 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:12.356265 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7832885-4cf6-4fcd-8c4d-f4be32a04300 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:12.366793 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1188/2390] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:12.940189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3e8d6-b50b-49e5-a985-8c7b8ceef625 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:12.955096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:13.055905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3e8d6-b50b-49e5-a985-8c7b8ceef625 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:13.061987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:13.063006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:13.063369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:13.063847 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] PATH_INFO: `/v3/users/bdd5a23501b748c3b43f17e04684057a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:13.065213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdd5a23501b748c3b43f17e04684057a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:13.068007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:13.145822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdd5a23501b748c3b43f17e04684057a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:13.156849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdd5a23501b748c3b43f17e04684057a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:13.157694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdd5a23501b748c3b43f17e04684057a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:13.171662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdd5a23501b748c3b43f17e04684057a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:13.178675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdd5a23501b748c3b43f17e04684057a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:13.186468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1203/2391] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:12 2022] DELETE /identity/v3/users/bdd5a23501b748c3b43f17e04684057a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:13.190925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e75b04-1807-46eb-8eb4-20df706cf1ef None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:13.191838 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e75b04-1807-46eb-8eb4-20df706cf1ef None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:13.192303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e75b04-1807-46eb-8eb4-20df706cf1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:13.215093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-03e75b04-1807-46eb-8eb4-20df706cf1ef None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:13.217189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-03e75b04-1807-46eb-8eb4-20df706cf1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:13.295810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1189/2392] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:13.311299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:13.313796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:13.384404 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592327b-b088-4021-b101-6917b4698215 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:13.390783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5487c1-675c-48d1-81b2-887c4cfa9898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:13.391739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5487c1-675c-48d1-81b2-887c4cfa9898 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:13.391873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5487c1-675c-48d1-81b2-887c4cfa9898 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:13.392588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5487c1-675c-48d1-81b2-887c4cfa9898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:13.405143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5487c1-675c-48d1-81b2-887c4cfa9898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:13.406672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5487c1-675c-48d1-81b2-887c4cfa9898 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:13.419361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1204/2393] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:14.000817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523451e-ce54-495e-b707-a144bd581cfa None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:14.001235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523451e-ce54-495e-b707-a144bd581cfa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:14.001821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523451e-ce54-495e-b707-a144bd581cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:14.024764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a523451e-ce54-495e-b707-a144bd581cfa None None] MFA Rules not processed for user `dcb88983a8e1439588644b71170d7362`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:14.026391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a523451e-ce54-495e-b707-a144bd581cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:14.040626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:14.063236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5487c1-675c-48d1-81b2-887c4cfa9898 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:14.065211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6f5487c1-675c-48d1-81b2-887c4cfa9898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:14.096363 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5487c1-675c-48d1-81b2-887c4cfa9898 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:14.104800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:14.105477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:14.105761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:14.106034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] PATH_INFO: `/v3/projects/b348a934b7564433ae70811baedc88d1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:14.107676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b348a934b7564433ae70811baedc88d1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:14.110237 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:14.144761 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1190/2394] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:21:14.175745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:14.214285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b348a934b7564433ae70811baedc88d1', 'request_id': 'req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:14.239722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b348a934b7564433ae70811baedc88d1', 'request_id': 'req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:14.253014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b348a934b7564433ae70811baedc88d1', 'request_id': 'req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:14.272879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] The token cache is being invalidate because project b348a934b7564433ae70811baedc88d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:14.273456 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:14.274323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:14.277131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1205/2395] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:14 2022] DELETE /identity/v3/projects/b348a934b7564433ae70811baedc88d1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:21:14.281211 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523451e-ce54-495e-b707-a144bd581cfa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:14.293696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523451e-ce54-495e-b707-a144bd581cfa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:14.360623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b78f01-9e68-4d20-8768-885de69de85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:14.361993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b78f01-9e68-4d20-8768-885de69de85d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:14.362402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b78f01-9e68-4d20-8768-885de69de85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:14.362752 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b78f01-9e68-4d20-8768-885de69de85d admin admin] PATH_INFO: `/v3/users/1aa002f9f49544b1b57eef77d79c61e7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:14.364617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b78f01-9e68-4d20-8768-885de69de85d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aa002f9f49544b1b57eef77d79c61e7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:14.367867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b78f01-9e68-4d20-8768-885de69de85d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:14.424931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e0b78f01-9e68-4d20-8768-885de69de85d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aa002f9f49544b1b57eef77d79c61e7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:14.435699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e0b78f01-9e68-4d20-8768-885de69de85d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aa002f9f49544b1b57eef77d79c61e7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:14.450231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e0b78f01-9e68-4d20-8768-885de69de85d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aa002f9f49544b1b57eef77d79c61e7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:14.451685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e0b78f01-9e68-4d20-8768-885de69de85d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aa002f9f49544b1b57eef77d79c61e7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:14.459792 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e0b78f01-9e68-4d20-8768-885de69de85d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aa002f9f49544b1b57eef77d79c61e7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:14.470464 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1191/2396] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:14 2022] DELETE /identity/v3/users/1aa002f9f49544b1b57eef77d79c61e7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:14.808248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:14.809703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:14.823852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefe3f0-bb1b-44eb-b76d-2e521cb8e835 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:14.902012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f129c-0a53-48cd-9dd6-1fec4107ab41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:14.902834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f129c-0a53-48cd-9dd6-1fec4107ab41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:14.903261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f129c-0a53-48cd-9dd6-1fec4107ab41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:14.903633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f129c-0a53-48cd-9dd6-1fec4107ab41 admin admin] PATH_INFO: `/v3/projects/aedc3c92dd534d26be3eea7be6f3ae72` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:14.905639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f129c-0a53-48cd-9dd6-1fec4107ab41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aedc3c92dd534d26be3eea7be6f3ae72)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:14.907756 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f129c-0a53-48cd-9dd6-1fec4107ab41 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:14.979315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-014f129c-0a53-48cd-9dd6-1fec4107ab41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aedc3c92dd534d26be3eea7be6f3ae72', 'request_id': 'req-014f129c-0a53-48cd-9dd6-1fec4107ab41'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:14.988730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-014f129c-0a53-48cd-9dd6-1fec4107ab41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aedc3c92dd534d26be3eea7be6f3ae72', 'request_id': 'req-014f129c-0a53-48cd-9dd6-1fec4107ab41'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:14.995516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-014f129c-0a53-48cd-9dd6-1fec4107ab41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aedc3c92dd534d26be3eea7be6f3ae72', 'request_id': 'req-014f129c-0a53-48cd-9dd6-1fec4107ab41'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:15.007865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-014f129c-0a53-48cd-9dd6-1fec4107ab41 admin admin] The token cache is being invalidate because project aedc3c92dd534d26be3eea7be6f3ae72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:15.008239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-014f129c-0a53-48cd-9dd6-1fec4107ab41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:15.009645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-014f129c-0a53-48cd-9dd6-1fec4107ab41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:15.011481 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1206/2397] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:14 2022] DELETE /identity/v3/projects/aedc3c92dd534d26be3eea7be6f3ae72 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:15.016946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91090e5-b257-4af7-8fe8-7547fe7b7404 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.017225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91090e5-b257-4af7-8fe8-7547fe7b7404 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.017513 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91090e5-b257-4af7-8fe8-7547fe7b7404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.037818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d91090e5-b257-4af7-8fe8-7547fe7b7404 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:15.039028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d91090e5-b257-4af7-8fe8-7547fe7b7404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:15.074943 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:15.107224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1192/2398] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:15.116508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e3597-c250-4bcf-9eb8-266991ae287e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.117041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e3597-c250-4bcf-9eb8-266991ae287e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.117155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e3597-c250-4bcf-9eb8-266991ae287e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.139789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4d5e3597-c250-4bcf-9eb8-266991ae287e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:15.141009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4d5e3597-c250-4bcf-9eb8-266991ae287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:15.149892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1207/2399] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:15.153825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91090e5-b257-4af7-8fe8-7547fe7b7404 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:15.164116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91090e5-b257-4af7-8fe8-7547fe7b7404 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:15.168871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db7e20-61f7-499e-bfbd-d2c710c46dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:15.169641 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db7e20-61f7-499e-bfbd-d2c710c46dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.170072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db7e20-61f7-499e-bfbd-d2c710c46dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.170354 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db7e20-61f7-499e-bfbd-d2c710c46dd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.171008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db7e20-61f7-499e-bfbd-d2c710c46dd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:21:15.171350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db7e20-61f7-499e-bfbd-d2c710c46dd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:15.172959 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db7e20-61f7-499e-bfbd-d2c710c46dd8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:15.185447 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1193/2400] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:21:15.190152 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e3597-c250-4bcf-9eb8-266991ae287e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:15.199866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e3597-c250-4bcf-9eb8-266991ae287e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:15.205722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8ff17-b12f-480c-bf89-98525b124d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:15.206326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8ff17-b12f-480c-bf89-98525b124d10 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.206602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8ff17-b12f-480c-bf89-98525b124d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.206871 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8ff17-b12f-480c-bf89-98525b124d10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.207543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8ff17-b12f-480c-bf89-98525b124d10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:15.209482 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8ff17-b12f-480c-bf89-98525b124d10 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:15.237965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1208/2401] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:15 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:21:15.242193 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db7e20-61f7-499e-bfbd-d2c710c46dd8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:15.253227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db7e20-61f7-499e-bfbd-d2c710c46dd8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:15.315875 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b68f7-34c4-4b16-a9a1-e06d3460a72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:15.316511 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b68f7-34c4-4b16-a9a1-e06d3460a72b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.316808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b68f7-34c4-4b16-a9a1-e06d3460a72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.317083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b68f7-34c4-4b16-a9a1-e06d3460a72b admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.317711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b68f7-34c4-4b16-a9a1-e06d3460a72b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:15.319688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b68f7-34c4-4b16-a9a1-e06d3460a72b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:15.343631 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1194/2402] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:15 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:15.347864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8ff17-b12f-480c-bf89-98525b124d10 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:15.365557 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8ff17-b12f-480c-bf89-98525b124d10 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:15.370653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43414ea2-cf6d-4133-ad95-48a10a2ffeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:15.371319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43414ea2-cf6d-4133-ad95-48a10a2ffeeb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.371883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43414ea2-cf6d-4133-ad95-48a10a2ffeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.372158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43414ea2-cf6d-4133-ad95-48a10a2ffeeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.373128 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43414ea2-cf6d-4133-ad95-48a10a2ffeeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:15.375142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43414ea2-cf6d-4133-ad95-48a10a2ffeeb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:15.390028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1209/2403] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:15.391398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b68f7-34c4-4b16-a9a1-e06d3460a72b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:15.401748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b68f7-34c4-4b16-a9a1-e06d3460a72b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:15.407390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c15d3-4173-43a7-af75-15f2abd0bafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:15.408102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c15d3-4173-43a7-af75-15f2abd0bafa admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.408379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c15d3-4173-43a7-af75-15f2abd0bafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.408739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c15d3-4173-43a7-af75-15f2abd0bafa admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.409378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c15d3-4173-43a7-af75-15f2abd0bafa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:15.411010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c15d3-4173-43a7-af75-15f2abd0bafa admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:15.423094 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1195/2404] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:15.432532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43414ea2-cf6d-4133-ad95-48a10a2ffeeb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:15.440285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:15.444478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43414ea2-cf6d-4133-ad95-48a10a2ffeeb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:15.452062 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e357c79-25d3-4f66-8811-0463fdc82f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:15.453148 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e357c79-25d3-4f66-8811-0463fdc82f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.453489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e357c79-25d3-4f66-8811-0463fdc82f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.453938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e357c79-25d3-4f66-8811-0463fdc82f67 admin admin] PATH_INFO: `/v3/projects/cb13a4d2a9104398bad60466dd7e0c9e/users/c5c5e8efe6944b7eaaf74d3db93163fa/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.469820 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e357c79-25d3-4f66-8811-0463fdc82f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb13a4d2a9104398bad60466dd7e0c9e, user_id=c5c5e8efe6944b7eaaf74d3db93163fa, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:15.472652 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e357c79-25d3-4f66-8811-0463fdc82f67 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:15.486802 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1210/2405] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:15 2022] PUT /identity/v3/projects/cb13a4d2a9104398bad60466dd7e0c9e/users/c5c5e8efe6944b7eaaf74d3db93163fa/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:15.499112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6cbd3-498c-4bc2-b843-30c96908d3e0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.499464 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6cbd3-498c-4bc2-b843-30c96908d3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.499818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6cbd3-498c-4bc2-b843-30c96908d3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.537138 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-05b6cbd3-498c-4bc2-b843-30c96908d3e0 None None] MFA Rules not processed for user `c5c5e8efe6944b7eaaf74d3db93163fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:15.538690 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-05b6cbd3-498c-4bc2-b843-30c96908d3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:15.609475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1196/2406] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:15.614043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e357c79-25d3-4f66-8811-0463fdc82f67 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:15.679751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e357c79-25d3-4f66-8811-0463fdc82f67 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:15.684466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a594e2-ec9c-48bc-b1dc-363a27d425b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:15.685147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a594e2-ec9c-48bc-b1dc-363a27d425b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.685425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a594e2-ec9c-48bc-b1dc-363a27d425b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.685698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a594e2-ec9c-48bc-b1dc-363a27d425b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.686325 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a594e2-ec9c-48bc-b1dc-363a27d425b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:15.688310 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a594e2-ec9c-48bc-b1dc-363a27d425b5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:15.717981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1211/2407] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:15 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:15.722028 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6cbd3-498c-4bc2-b843-30c96908d3e0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:15.732797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6cbd3-498c-4bc2-b843-30c96908d3e0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:15.762017 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:15.793072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a6cfc-5eac-4178-bdea-66c940b4fda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:15.793701 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a6cfc-5eac-4178-bdea-66c940b4fda4 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.793975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a6cfc-5eac-4178-bdea-66c940b4fda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.794238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a6cfc-5eac-4178-bdea-66c940b4fda4 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.795317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a6cfc-5eac-4178-bdea-66c940b4fda4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:15.797270 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a6cfc-5eac-4178-bdea-66c940b4fda4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:15.817956 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1197/2408] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:15 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:15.821991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a594e2-ec9c-48bc-b1dc-363a27d425b5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:15.833239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a594e2-ec9c-48bc-b1dc-363a27d425b5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:15.838060 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91416e82-c0a9-4770-a7aa-83933bf56f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:15.838725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91416e82-c0a9-4770-a7aa-83933bf56f4b admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.839062 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91416e82-c0a9-4770-a7aa-83933bf56f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.839399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91416e82-c0a9-4770-a7aa-83933bf56f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.840099 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91416e82-c0a9-4770-a7aa-83933bf56f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:15.842969 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91416e82-c0a9-4770-a7aa-83933bf56f4b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:15.853695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1212/2409] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:15.858227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a6cfc-5eac-4178-bdea-66c940b4fda4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:15.868191 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a6cfc-5eac-4178-bdea-66c940b4fda4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:15.873039 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9949957-f026-4186-89b9-cc1b8bb74a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:15.873683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9949957-f026-4186-89b9-cc1b8bb74a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.873965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9949957-f026-4186-89b9-cc1b8bb74a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.874238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9949957-f026-4186-89b9-cc1b8bb74a52 admin admin] PATH_INFO: `/v3/projects/542badfb74ba4308b62f538e1b1d02fd/users/ae25462830a645e48b1ceddf4306a368/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.889250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9949957-f026-4186-89b9-cc1b8bb74a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=542badfb74ba4308b62f538e1b1d02fd, user_id=ae25462830a645e48b1ceddf4306a368, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:15.891883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9949957-f026-4186-89b9-cc1b8bb74a52 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:15.904810 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1198/2410] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:15 2022] PUT /identity/v3/projects/542badfb74ba4308b62f538e1b1d02fd/users/ae25462830a645e48b1ceddf4306a368/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:15.908814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91416e82-c0a9-4770-a7aa-83933bf56f4b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:15.918943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91416e82-c0a9-4770-a7aa-83933bf56f4b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:15.978192 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeb19c-64f6-4121-a628-a9f5afcd6cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:15.978804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeb19c-64f6-4121-a628-a9f5afcd6cfe admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:15.979118 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeb19c-64f6-4121-a628-a9f5afcd6cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:15.979395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeb19c-64f6-4121-a628-a9f5afcd6cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:15.980036 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aeb19c-64f6-4121-a628-a9f5afcd6cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:15.981645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aeb19c-64f6-4121-a628-a9f5afcd6cfe admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:15.993060 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1213/2411] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:15.998035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9949957-f026-4186-89b9-cc1b8bb74a52 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:16.023879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9949957-f026-4186-89b9-cc1b8bb74a52 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:16.029771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaf208-288b-47db-805d-2706c8d98341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:16.030532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf208-288b-47db-805d-2706c8d98341 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:16.030879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf208-288b-47db-805d-2706c8d98341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:16.031248 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf208-288b-47db-805d-2706c8d98341 admin admin] PATH_INFO: `/v3/projects/542badfb74ba4308b62f538e1b1d02fd/users/ae25462830a645e48b1ceddf4306a368/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:16.034309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaf208-288b-47db-805d-2706c8d98341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=542badfb74ba4308b62f538e1b1d02fd, user_id=ae25462830a645e48b1ceddf4306a368, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:16.037123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaf208-288b-47db-805d-2706c8d98341 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:16.051083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1199/2412] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:15 2022] PUT /identity/v3/projects/542badfb74ba4308b62f538e1b1d02fd/users/ae25462830a645e48b1ceddf4306a368/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:16.062706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10091b-3935-4fb1-90c9-394d0343e894 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:16.063348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10091b-3935-4fb1-90c9-394d0343e894 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:16.063921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10091b-3935-4fb1-90c9-394d0343e894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:16.101085 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0c10091b-3935-4fb1-90c9-394d0343e894 None None] MFA Rules not processed for user `ae25462830a645e48b1ceddf4306a368`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:16.101966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0c10091b-3935-4fb1-90c9-394d0343e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:16.153790 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:16.186968 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1214/2413] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:16.198637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaf208-288b-47db-805d-2706c8d98341 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:16.201601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5bbaf208-288b-47db-805d-2706c8d98341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:16.263943 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaf208-288b-47db-805d-2706c8d98341 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:16.267650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0782e-231d-4653-8ba1-1d164d70fd7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:16.268256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0782e-231d-4653-8ba1-1d164d70fd7a service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:16.268535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0782e-231d-4653-8ba1-1d164d70fd7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:16.269034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0782e-231d-4653-8ba1-1d164d70fd7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:16.278016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0782e-231d-4653-8ba1-1d164d70fd7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:16.279525 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0782e-231d-4653-8ba1-1d164d70fd7a service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:16.288857 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1200/2414] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:16.370912 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10091b-3935-4fb1-90c9-394d0343e894 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:16.382580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10091b-3935-4fb1-90c9-394d0343e894 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:16.386931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b69846-2d07-41b3-be8f-32007df91654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:16.387637 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b69846-2d07-41b3-be8f-32007df91654 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:16.387848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b69846-2d07-41b3-be8f-32007df91654 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:16.388138 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b69846-2d07-41b3-be8f-32007df91654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:16.455025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b69846-2d07-41b3-be8f-32007df91654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:16.456715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b69846-2d07-41b3-be8f-32007df91654 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:16.466108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1215/2415] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:21:25.797061 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0782e-231d-4653-8ba1-1d164d70fd7a service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:25.866855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0782e-231d-4653-8ba1-1d164d70fd7a service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:25.871583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:25.872496 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:25.872778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:25.873054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] PATH_INFO: `/v3/users/c5c5e8efe6944b7eaaf74d3db93163fa` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:25.874753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5c5e8efe6944b7eaaf74d3db93163fa)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:25.877037 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:25.912911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:25.932672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5c5e8efe6944b7eaaf74d3db93163fa'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:25.942590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5c5e8efe6944b7eaaf74d3db93163fa'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:25.958105 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5c5e8efe6944b7eaaf74d3db93163fa'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:25.959333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5c5e8efe6944b7eaaf74d3db93163fa'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:25.978592 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5c5e8efe6944b7eaaf74d3db93163fa'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:25.985317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1201/2416] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:25 2022] DELETE /identity/v3/users/c5c5e8efe6944b7eaaf74d3db93163fa => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:25.989880 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23bb8c-8f95-4a29-bdee-fadcb59a7c62 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:25.990380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23bb8c-8f95-4a29-bdee-fadcb59a7c62 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:25.990523 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23bb8c-8f95-4a29-bdee-fadcb59a7c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:26.010390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9d23bb8c-8f95-4a29-bdee-fadcb59a7c62 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:26.011575 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9d23bb8c-8f95-4a29-bdee-fadcb59a7c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:26.073364 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1216/2417] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:26.084564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:26.086296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:26.144567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f427b-8d50-4b9d-a5a8-455b38fae71e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:26.148502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25811521-f764-40bd-90e6-de07e1716f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:26.149226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25811521-f764-40bd-90e6-de07e1716f5f service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:26.149605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25811521-f764-40bd-90e6-de07e1716f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:26.149975 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25811521-f764-40bd-90e6-de07e1716f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:26.159488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25811521-f764-40bd-90e6-de07e1716f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:26.160938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25811521-f764-40bd-90e6-de07e1716f5f service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:26.171011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1202/2418] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:26.465765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23bb8c-8f95-4a29-bdee-fadcb59a7c62 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:26.468521 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9d23bb8c-8f95-4a29-bdee-fadcb59a7c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:26.481366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23bb8c-8f95-4a29-bdee-fadcb59a7c62 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:26.488391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:26.493939 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:26.494230 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:26.494510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] PATH_INFO: `/v3/projects/cb13a4d2a9104398bad60466dd7e0c9e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:26.495833 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb13a4d2a9104398bad60466dd7e0c9e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:26.498104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:26.554585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb13a4d2a9104398bad60466dd7e0c9e', 'request_id': 'req-8ed97599-5bae-4f68-b4d3-210c149993e8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:26.564545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb13a4d2a9104398bad60466dd7e0c9e', 'request_id': 'req-8ed97599-5bae-4f68-b4d3-210c149993e8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:26.576494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb13a4d2a9104398bad60466dd7e0c9e', 'request_id': 'req-8ed97599-5bae-4f68-b4d3-210c149993e8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:26.585063 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:26.590653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] The token cache is being invalidate because project cb13a4d2a9104398bad60466dd7e0c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:26.591103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:26.593607 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:26.598573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1217/2419] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:26 2022] DELETE /identity/v3/projects/cb13a4d2a9104398bad60466dd7e0c9e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:26.600077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25811521-f764-40bd-90e6-de07e1716f5f service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:26.612294 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25811521-f764-40bd-90e6-de07e1716f5f service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:26.676638 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ea511-5a26-416b-b7e2-c80083097365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:26.677333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ea511-5a26-416b-b7e2-c80083097365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:26.677615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ea511-5a26-416b-b7e2-c80083097365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:26.677889 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ea511-5a26-416b-b7e2-c80083097365 admin admin] PATH_INFO: `/v3/users/ae25462830a645e48b1ceddf4306a368` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:26.679145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ea511-5a26-416b-b7e2-c80083097365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae25462830a645e48b1ceddf4306a368)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:26.681309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ea511-5a26-416b-b7e2-c80083097365 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:26.736487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-694ea511-5a26-416b-b7e2-c80083097365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae25462830a645e48b1ceddf4306a368'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:26.743794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:26.751911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-694ea511-5a26-416b-b7e2-c80083097365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae25462830a645e48b1ceddf4306a368'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:26.770137 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-694ea511-5a26-416b-b7e2-c80083097365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae25462830a645e48b1ceddf4306a368'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:26.770890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-694ea511-5a26-416b-b7e2-c80083097365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae25462830a645e48b1ceddf4306a368'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:26.779225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-694ea511-5a26-416b-b7e2-c80083097365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae25462830a645e48b1ceddf4306a368'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:26.787102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1203/2420] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:26 2022] DELETE /identity/v3/users/ae25462830a645e48b1ceddf4306a368 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:27.405117 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:27.406619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:27.424156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed97599-5bae-4f68-b4d3-210c149993e8 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:27.518841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94b294-f609-41c1-b795-16a19fbae13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:27.519524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94b294-f609-41c1-b795-16a19fbae13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:27.520189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94b294-f609-41c1-b795-16a19fbae13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:27.521064 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94b294-f609-41c1-b795-16a19fbae13b admin admin] PATH_INFO: `/v3/projects/542badfb74ba4308b62f538e1b1d02fd` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:27.522795 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94b294-f609-41c1-b795-16a19fbae13b admin admin] RBAC: Authorizing `identity:delete_project(project_id=542badfb74ba4308b62f538e1b1d02fd)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:27.526558 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94b294-f609-41c1-b795-16a19fbae13b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:27.597785 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8d94b294-f609-41c1-b795-16a19fbae13b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '542badfb74ba4308b62f538e1b1d02fd', 'request_id': 'req-8d94b294-f609-41c1-b795-16a19fbae13b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:27.610567 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8d94b294-f609-41c1-b795-16a19fbae13b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '542badfb74ba4308b62f538e1b1d02fd', 'request_id': 'req-8d94b294-f609-41c1-b795-16a19fbae13b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:27.620799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8d94b294-f609-41c1-b795-16a19fbae13b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '542badfb74ba4308b62f538e1b1d02fd', 'request_id': 'req-8d94b294-f609-41c1-b795-16a19fbae13b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:27.635811 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8d94b294-f609-41c1-b795-16a19fbae13b admin admin] The token cache is being invalidate because project 542badfb74ba4308b62f538e1b1d02fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:27.636107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8d94b294-f609-41c1-b795-16a19fbae13b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:27.636798 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8d94b294-f609-41c1-b795-16a19fbae13b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:27.638601 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1218/2421] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:27 2022] DELETE /identity/v3/projects/542badfb74ba4308b62f538e1b1d02fd => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:27.646967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48b11a-41ce-4069-a0cd-992dbe1a36b4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:27.647553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48b11a-41ce-4069-a0cd-992dbe1a36b4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:27.647553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48b11a-41ce-4069-a0cd-992dbe1a36b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:27.670621 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-dc48b11a-41ce-4069-a0cd-992dbe1a36b4 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:27.672238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dc48b11a-41ce-4069-a0cd-992dbe1a36b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:27.758350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1204/2422] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:27.760104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b566a6-49a9-403f-8344-b4708b5880da None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:27.760392 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b566a6-49a9-403f-8344-b4708b5880da None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:27.760662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b566a6-49a9-403f-8344-b4708b5880da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:27.786572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-21b566a6-49a9-403f-8344-b4708b5880da None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:27.787898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-21b566a6-49a9-403f-8344-b4708b5880da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:27.796872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1219/2423] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:27.801139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48b11a-41ce-4069-a0cd-992dbe1a36b4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:27.812687 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48b11a-41ce-4069-a0cd-992dbe1a36b4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:27.817817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98908e04-47df-472c-982b-39db9bbea639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:27.818549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908e04-47df-472c-982b-39db9bbea639 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:27.818905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908e04-47df-472c-982b-39db9bbea639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:27.819422 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908e04-47df-472c-982b-39db9bbea639 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:27.820270 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98908e04-47df-472c-982b-39db9bbea639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:21:27.820587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98908e04-47df-472c-982b-39db9bbea639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:27.822873 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98908e04-47df-472c-982b-39db9bbea639 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:27.837417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1205/2424] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:21:27.842096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b566a6-49a9-403f-8344-b4708b5880da None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:27.855299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b566a6-49a9-403f-8344-b4708b5880da None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:27.867003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3d67a-1c78-4c53-9787-e64b79a287ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:27.867896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3d67a-1c78-4c53-9787-e64b79a287ff admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:27.868689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3d67a-1c78-4c53-9787-e64b79a287ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:27.868975 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3d67a-1c78-4c53-9787-e64b79a287ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:27.869926 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3d67a-1c78-4c53-9787-e64b79a287ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:27.872186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3d67a-1c78-4c53-9787-e64b79a287ff admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:27.908229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1220/2425] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:27 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:27.909540 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98908e04-47df-472c-982b-39db9bbea639 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:27.922148 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98908e04-47df-472c-982b-39db9bbea639 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:27.984346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008d61b-cf36-4672-bed4-826b9447d393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:27.984976 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008d61b-cf36-4672-bed4-826b9447d393 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:27.985313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008d61b-cf36-4672-bed4-826b9447d393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:27.985588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008d61b-cf36-4672-bed4-826b9447d393 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:27.986230 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008d61b-cf36-4672-bed4-826b9447d393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:27.988257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008d61b-cf36-4672-bed4-826b9447d393 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:28.014078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1206/2426] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:27 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:28.015960 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3d67a-1c78-4c53-9787-e64b79a287ff admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:28.026597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3d67a-1c78-4c53-9787-e64b79a287ff admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:28.031584 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294995ff-fc78-45ed-b649-d839ea6ecc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:28.032222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294995ff-fc78-45ed-b649-d839ea6ecc76 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:28.032488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294995ff-fc78-45ed-b649-d839ea6ecc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:28.032758 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294995ff-fc78-45ed-b649-d839ea6ecc76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:28.033484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294995ff-fc78-45ed-b649-d839ea6ecc76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:28.035120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294995ff-fc78-45ed-b649-d839ea6ecc76 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:28.046643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1221/2427] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:21:28.050874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008d61b-cf36-4672-bed4-826b9447d393 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:28.061548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008d61b-cf36-4672-bed4-826b9447d393 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:28.066436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d6b1f-0b3b-41d9-93f3-c00023e52d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:28.067095 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d6b1f-0b3b-41d9-93f3-c00023e52d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:28.067376 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d6b1f-0b3b-41d9-93f3-c00023e52d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:28.067647 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d6b1f-0b3b-41d9-93f3-c00023e52d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:28.068297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d6b1f-0b3b-41d9-93f3-c00023e52d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:28.069913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d6b1f-0b3b-41d9-93f3-c00023e52d73 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:28.086290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1207/2428] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:21:28.087847 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294995ff-fc78-45ed-b649-d839ea6ecc76 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:28.107343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294995ff-fc78-45ed-b649-d839ea6ecc76 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:28.113039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad5c97-982f-49c4-85f7-4310d368d88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:28.114100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad5c97-982f-49c4-85f7-4310d368d88f admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:28.115516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad5c97-982f-49c4-85f7-4310d368d88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:28.115975 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad5c97-982f-49c4-85f7-4310d368d88f admin admin] PATH_INFO: `/v3/projects/75dc303b7a8c4c848d4a3fb6166ca400/users/035b0c2135114db7a9320e7ac9ddb472/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:28.134532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad5c97-982f-49c4-85f7-4310d368d88f admin admin] RBAC: Authorizing `identity:create_grant(project_id=75dc303b7a8c4c848d4a3fb6166ca400, user_id=035b0c2135114db7a9320e7ac9ddb472, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:28.146502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad5c97-982f-49c4-85f7-4310d368d88f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:28.162248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1222/2429] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:28 2022] PUT /identity/v3/projects/75dc303b7a8c4c848d4a3fb6166ca400/users/035b0c2135114db7a9320e7ac9ddb472/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:28.172725 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b605b7-6b8d-4cf7-a7db-a1a110842d60 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:28.173015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b605b7-6b8d-4cf7-a7db-a1a110842d60 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:28.173292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b605b7-6b8d-4cf7-a7db-a1a110842d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:28.202325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-59b605b7-6b8d-4cf7-a7db-a1a110842d60 None None] MFA Rules not processed for user `035b0c2135114db7a9320e7ac9ddb472`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:28.203629 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-59b605b7-6b8d-4cf7-a7db-a1a110842d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:28.227573 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:28.274252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1208/2430] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:21:28.278728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad5c97-982f-49c4-85f7-4310d368d88f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:28.307578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:28.345610 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad5c97-982f-49c4-85f7-4310d368d88f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:28.350578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b91f5-0e84-4e12-b69a-8c7955cfbbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:28.351218 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b91f5-0e84-4e12-b69a-8c7955cfbbdd admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:28.351499 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b91f5-0e84-4e12-b69a-8c7955cfbbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:28.351772 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b91f5-0e84-4e12-b69a-8c7955cfbbdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:28.352431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b91f5-0e84-4e12-b69a-8c7955cfbbdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:28.354388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b91f5-0e84-4e12-b69a-8c7955cfbbdd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:28.385407 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1223/2431] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:28 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:28.386521 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b605b7-6b8d-4cf7-a7db-a1a110842d60 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:28.397412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b605b7-6b8d-4cf7-a7db-a1a110842d60 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:28.456931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b13f37-6757-4dcf-b0ba-31d0261d74fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:28.457697 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b13f37-6757-4dcf-b0ba-31d0261d74fd admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:28.457985 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b13f37-6757-4dcf-b0ba-31d0261d74fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:28.458261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b13f37-6757-4dcf-b0ba-31d0261d74fd admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:28.458892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b13f37-6757-4dcf-b0ba-31d0261d74fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:28.460896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b13f37-6757-4dcf-b0ba-31d0261d74fd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:28.486038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1209/2432] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:28 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:21:28.487281 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b91f5-0e84-4e12-b69a-8c7955cfbbdd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:28.497782 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b91f5-0e84-4e12-b69a-8c7955cfbbdd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:28.503596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a7d18-421f-4fe9-9597-c7f882be582b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:28.504346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a7d18-421f-4fe9-9597-c7f882be582b admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:28.504645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a7d18-421f-4fe9-9597-c7f882be582b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:28.504897 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a7d18-421f-4fe9-9597-c7f882be582b admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:28.505628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a7d18-421f-4fe9-9597-c7f882be582b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:28.507571 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a7d18-421f-4fe9-9597-c7f882be582b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:28.522340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1224/2433] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:21:28.523480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b13f37-6757-4dcf-b0ba-31d0261d74fd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:28.533899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b13f37-6757-4dcf-b0ba-31d0261d74fd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:28.539272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4efba0-b0fc-4e73-b842-1edcbbcd5ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:28.539942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4efba0-b0fc-4e73-b842-1edcbbcd5ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:28.540223 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4efba0-b0fc-4e73-b842-1edcbbcd5ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:28.540646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4efba0-b0fc-4e73-b842-1edcbbcd5ea5 admin admin] PATH_INFO: `/v3/projects/9134f4560497413095b918b62def3cd2/users/bfdc12c7e04443dca6dece9c09e2da9b/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:28.562325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4efba0-b0fc-4e73-b842-1edcbbcd5ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9134f4560497413095b918b62def3cd2, user_id=bfdc12c7e04443dca6dece9c09e2da9b, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:28.565097 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4efba0-b0fc-4e73-b842-1edcbbcd5ea5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:28.572714 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a7d18-421f-4fe9-9597-c7f882be582b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:28.574920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f41a7d18-421f-4fe9-9597-c7f882be582b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:28.580100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1210/2434] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:28 2022] PUT /identity/v3/projects/9134f4560497413095b918b62def3cd2/users/bfdc12c7e04443dca6dece9c09e2da9b/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:21:28.583772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4efba0-b0fc-4e73-b842-1edcbbcd5ea5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:28.663412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4efba0-b0fc-4e73-b842-1edcbbcd5ea5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:28.668290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b02221-a8a7-4969-86da-0f97d15f8938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:28.668906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b02221-a8a7-4969-86da-0f97d15f8938 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:28.669213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b02221-a8a7-4969-86da-0f97d15f8938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:28.669465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b02221-a8a7-4969-86da-0f97d15f8938 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:28.670113 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b02221-a8a7-4969-86da-0f97d15f8938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:28.672680 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b02221-a8a7-4969-86da-0f97d15f8938 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:28.673893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a7d18-421f-4fe9-9597-c7f882be582b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:28.683289 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1211/2435] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:21:28.688320 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b02221-a8a7-4969-86da-0f97d15f8938 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:28.699760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b02221-a8a7-4969-86da-0f97d15f8938 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:28.706123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede3177-9bd5-4b3f-b573-3bfeb2b155d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:28.706801 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede3177-9bd5-4b3f-b573-3bfeb2b155d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:28.707297 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede3177-9bd5-4b3f-b573-3bfeb2b155d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:28.707585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede3177-9bd5-4b3f-b573-3bfeb2b155d3 admin admin] PATH_INFO: `/v3/projects/9134f4560497413095b918b62def3cd2/users/bfdc12c7e04443dca6dece9c09e2da9b/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:28.710235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede3177-9bd5-4b3f-b573-3bfeb2b155d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9134f4560497413095b918b62def3cd2, user_id=bfdc12c7e04443dca6dece9c09e2da9b, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:28.714049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede3177-9bd5-4b3f-b573-3bfeb2b155d3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:28.736769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1212/2436] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:28 2022] PUT /identity/v3/projects/9134f4560497413095b918b62def3cd2/users/bfdc12c7e04443dca6dece9c09e2da9b/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:28.748505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7150ad-99ab-4d21-89ed-7bdf0d409be6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:28.748814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7150ad-99ab-4d21-89ed-7bdf0d409be6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:28.749072 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7150ad-99ab-4d21-89ed-7bdf0d409be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:28.763337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cf91d-35b9-438b-a7ad-a3b5c01ac280 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:28.764089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cf91d-35b9-438b-a7ad-a3b5c01ac280 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:28.764454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cf91d-35b9-438b-a7ad-a3b5c01ac280 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:28.764827 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cf91d-35b9-438b-a7ad-a3b5c01ac280 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:28.766750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d26cf91d-35b9-438b-a7ad-a3b5c01ac280 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:28.777102 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:28.780871 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26cf91d-35b9-438b-a7ad-a3b5c01ac280 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:28.782566 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8a7150ad-99ab-4d21-89ed-7bdf0d409be6 None None] MFA Rules not processed for user `bfdc12c7e04443dca6dece9c09e2da9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:28.782884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26cf91d-35b9-438b-a7ad-a3b5c01ac280 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:28.785032 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8a7150ad-99ab-4d21-89ed-7bdf0d409be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:28.795136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1225/2437] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:21:28.866273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1213/2438] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:21:28.878236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cf91d-35b9-438b-a7ad-a3b5c01ac280 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:28.880301 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d26cf91d-35b9-438b-a7ad-a3b5c01ac280 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:28.918764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:28.959723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cf91d-35b9-438b-a7ad-a3b5c01ac280 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:29.018342 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8a3b3-2523-4b51-9e3e-2173289288ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:29.018977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8a3b3-2523-4b51-9e3e-2173289288ca service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:29.019312 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8a3b3-2523-4b51-9e3e-2173289288ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:29.019738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8a3b3-2523-4b51-9e3e-2173289288ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:29.029889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8a3b3-2523-4b51-9e3e-2173289288ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:29.031889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8a3b3-2523-4b51-9e3e-2173289288ca service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:29.042250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1226/2439] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:29.091290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7150ad-99ab-4d21-89ed-7bdf0d409be6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:29.092928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8a7150ad-99ab-4d21-89ed-7bdf0d409be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:29.105298 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7150ad-99ab-4d21-89ed-7bdf0d409be6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:29.109908 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409e17d-d13e-415d-830d-e773dc744c83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:29.110602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409e17d-d13e-415d-830d-e773dc744c83 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:29.110935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409e17d-d13e-415d-830d-e773dc744c83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:29.111296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409e17d-d13e-415d-830d-e773dc744c83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:29.174212 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409e17d-d13e-415d-830d-e773dc744c83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:29.176851 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409e17d-d13e-415d-830d-e773dc744c83 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:29.186772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1214/2440] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:32.578317 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8a3b3-2523-4b51-9e3e-2173289288ca service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:32.648422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8a3b3-2523-4b51-9e3e-2173289288ca service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:32.653142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:32.653804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:32.654083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:32.654355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] PATH_INFO: `/v3/users/035b0c2135114db7a9320e7ac9ddb472` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:32.655879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] RBAC: Authorizing `identity:delete_user(user_id=035b0c2135114db7a9320e7ac9ddb472)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:32.657796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:32.706266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '035b0c2135114db7a9320e7ac9ddb472'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:32.713588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '035b0c2135114db7a9320e7ac9ddb472'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:32.714239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '035b0c2135114db7a9320e7ac9ddb472'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:32.726746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '035b0c2135114db7a9320e7ac9ddb472'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:32.733980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '035b0c2135114db7a9320e7ac9ddb472'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:32.736198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:32.740812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1227/2441] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:32 2022] DELETE /identity/v3/users/035b0c2135114db7a9320e7ac9ddb472 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:32.745451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c152618-58ac-43d3-b394-7871362dba70 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:32.745935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c152618-58ac-43d3-b394-7871362dba70 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:32.746033 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c152618-58ac-43d3-b394-7871362dba70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:32.766561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8c152618-58ac-43d3-b394-7871362dba70 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:32.767817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8c152618-58ac-43d3-b394-7871362dba70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:32.828712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1215/2442] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:32.838507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:32.839943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:32.911298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ee1da-d742-4a34-a3b9-eee96a43152b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:32.914971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb39818-d421-4723-a4e2-9af3e1b2386a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:32.915636 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb39818-d421-4723-a4e2-9af3e1b2386a service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:32.915915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb39818-d421-4723-a4e2-9af3e1b2386a service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:32.916185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb39818-d421-4723-a4e2-9af3e1b2386a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:32.926408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb39818-d421-4723-a4e2-9af3e1b2386a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:32.927872 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb39818-d421-4723-a4e2-9af3e1b2386a service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:32.938487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1228/2443] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:33.244664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c152618-58ac-43d3-b394-7871362dba70 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:33.246338 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8c152618-58ac-43d3-b394-7871362dba70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:33.261438 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c152618-58ac-43d3-b394-7871362dba70 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:33.266364 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:33.267027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:33.267366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:33.267677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] PATH_INFO: `/v3/projects/75dc303b7a8c4c848d4a3fb6166ca400` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:33.268852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=75dc303b7a8c4c848d4a3fb6166ca400)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:33.270925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:33.307362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:33.320316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75dc303b7a8c4c848d4a3fb6166ca400', 'request_id': 'req-a6fe98b9-7499-4e1e-a504-718d95852c9e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.325695 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75dc303b7a8c4c848d4a3fb6166ca400', 'request_id': 'req-a6fe98b9-7499-4e1e-a504-718d95852c9e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.331951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75dc303b7a8c4c848d4a3fb6166ca400', 'request_id': 'req-a6fe98b9-7499-4e1e-a504-718d95852c9e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.345953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] The token cache is being invalidate because project 75dc303b7a8c4c848d4a3fb6166ca400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:33.346318 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.347034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.348599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1216/2444] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:33 2022] DELETE /identity/v3/projects/75dc303b7a8c4c848d4a3fb6166ca400 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:33.352870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb39818-d421-4723-a4e2-9af3e1b2386a service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:33.363648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb39818-d421-4723-a4e2-9af3e1b2386a service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:33.422687 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca96e16-03e0-429f-8b54-62c3c1ad1e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:33.423367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca96e16-03e0-429f-8b54-62c3c1ad1e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:33.423650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca96e16-03e0-429f-8b54-62c3c1ad1e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:33.423980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca96e16-03e0-429f-8b54-62c3c1ad1e19 admin admin] PATH_INFO: `/v3/users/bfdc12c7e04443dca6dece9c09e2da9b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:33.425136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca96e16-03e0-429f-8b54-62c3c1ad1e19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfdc12c7e04443dca6dece9c09e2da9b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:33.427072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca96e16-03e0-429f-8b54-62c3c1ad1e19 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:33.482660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-eca96e16-03e0-429f-8b54-62c3c1ad1e19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfdc12c7e04443dca6dece9c09e2da9b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.491359 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-eca96e16-03e0-429f-8b54-62c3c1ad1e19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfdc12c7e04443dca6dece9c09e2da9b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.491973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-eca96e16-03e0-429f-8b54-62c3c1ad1e19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfdc12c7e04443dca6dece9c09e2da9b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.505518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-eca96e16-03e0-429f-8b54-62c3c1ad1e19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfdc12c7e04443dca6dece9c09e2da9b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.512413 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-eca96e16-03e0-429f-8b54-62c3c1ad1e19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfdc12c7e04443dca6dece9c09e2da9b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.518337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1229/2445] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:33 2022] DELETE /identity/v3/users/bfdc12c7e04443dca6dece9c09e2da9b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:33.804402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:33.805835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:33.820949 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe98b9-7499-4e1e-a504-718d95852c9e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:33.883676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4eab22-a446-42e0-a233-f937c6c7373d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:33.884306 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eab22-a446-42e0-a233-f937c6c7373d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:33.884580 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eab22-a446-42e0-a233-f937c6c7373d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:33.884848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eab22-a446-42e0-a233-f937c6c7373d admin admin] PATH_INFO: `/v3/projects/9134f4560497413095b918b62def3cd2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:33.885972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4eab22-a446-42e0-a233-f937c6c7373d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9134f4560497413095b918b62def3cd2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:33.887905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4eab22-a446-42e0-a233-f937c6c7373d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:33.939786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b4eab22-a446-42e0-a233-f937c6c7373d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9134f4560497413095b918b62def3cd2', 'request_id': 'req-6b4eab22-a446-42e0-a233-f937c6c7373d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.945584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b4eab22-a446-42e0-a233-f937c6c7373d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9134f4560497413095b918b62def3cd2', 'request_id': 'req-6b4eab22-a446-42e0-a233-f937c6c7373d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.952546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b4eab22-a446-42e0-a233-f937c6c7373d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9134f4560497413095b918b62def3cd2', 'request_id': 'req-6b4eab22-a446-42e0-a233-f937c6c7373d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.964400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b4eab22-a446-42e0-a233-f937c6c7373d admin admin] The token cache is being invalidate because project 9134f4560497413095b918b62def3cd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:33.964687 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b4eab22-a446-42e0-a233-f937c6c7373d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.965345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b4eab22-a446-42e0-a233-f937c6c7373d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:33.966794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1217/2446] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:33 2022] DELETE /identity/v3/projects/9134f4560497413095b918b62def3cd2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:33.972612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055e898-c315-4e14-a7eb-707f2432dab8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:33.973109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055e898-c315-4e14-a7eb-707f2432dab8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:33.973210 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055e898-c315-4e14-a7eb-707f2432dab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:33.993001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8055e898-c315-4e14-a7eb-707f2432dab8 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:33.994149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8055e898-c315-4e14-a7eb-707f2432dab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:34.012321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:34.058050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1230/2447] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:34.072751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c71f81-6f70-44d2-a91c-3bf339f69ad2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.073041 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c71f81-6f70-44d2-a91c-3bf339f69ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.077150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c71f81-6f70-44d2-a91c-3bf339f69ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.096561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-63c71f81-6f70-44d2-a91c-3bf339f69ad2 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:34.097769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-63c71f81-6f70-44d2-a91c-3bf339f69ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:34.105669 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1218/2448] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:21:34.110287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055e898-c315-4e14-a7eb-707f2432dab8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:34.120083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055e898-c315-4e14-a7eb-707f2432dab8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:34.125612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc53233-ec0b-4e2f-8cc7-0044e5acdbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:34.126208 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53233-ec0b-4e2f-8cc7-0044e5acdbf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.126487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53233-ec0b-4e2f-8cc7-0044e5acdbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.126761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53233-ec0b-4e2f-8cc7-0044e5acdbf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.127431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc53233-ec0b-4e2f-8cc7-0044e5acdbf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:21:34.127735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc53233-ec0b-4e2f-8cc7-0044e5acdbf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:34.129336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc53233-ec0b-4e2f-8cc7-0044e5acdbf6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:34.141462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1231/2449] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:21:34.145909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c71f81-6f70-44d2-a91c-3bf339f69ad2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:34.155413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c71f81-6f70-44d2-a91c-3bf339f69ad2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:34.160153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fb8b5-0a7d-4b96-a352-c43c72af0094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:34.160748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fb8b5-0a7d-4b96-a352-c43c72af0094 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.161020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fb8b5-0a7d-4b96-a352-c43c72af0094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.161289 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fb8b5-0a7d-4b96-a352-c43c72af0094 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.161922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fb8b5-0a7d-4b96-a352-c43c72af0094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:34.163874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fb8b5-0a7d-4b96-a352-c43c72af0094 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:34.191871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1219/2450] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:34 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:34.196270 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc53233-ec0b-4e2f-8cc7-0044e5acdbf6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:34.206957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc53233-ec0b-4e2f-8cc7-0044e5acdbf6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:34.265866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7e763-1580-4907-89ee-fea30f20f3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:34.266564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7e763-1580-4907-89ee-fea30f20f3cb admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.266842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7e763-1580-4907-89ee-fea30f20f3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.267147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7e763-1580-4907-89ee-fea30f20f3cb admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.267789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7e763-1580-4907-89ee-fea30f20f3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:34.269689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7e763-1580-4907-89ee-fea30f20f3cb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:34.290725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1232/2451] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:34 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:34.295117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fb8b5-0a7d-4b96-a352-c43c72af0094 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:34.305488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fb8b5-0a7d-4b96-a352-c43c72af0094 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:34.310202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79a68a-456b-43d6-b246-44d0f69a7366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:34.310789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79a68a-456b-43d6-b246-44d0f69a7366 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.311610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79a68a-456b-43d6-b246-44d0f69a7366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.311884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79a68a-456b-43d6-b246-44d0f69a7366 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.312549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79a68a-456b-43d6-b246-44d0f69a7366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:34.314151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79a68a-456b-43d6-b246-44d0f69a7366 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:34.324074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1220/2452] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:34.328625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7e763-1580-4907-89ee-fea30f20f3cb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:34.338486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7e763-1580-4907-89ee-fea30f20f3cb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:34.343218 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d246a-3a89-42ff-9b5e-ddcfd0f83aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:34.343817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d246a-3a89-42ff-9b5e-ddcfd0f83aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.344094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d246a-3a89-42ff-9b5e-ddcfd0f83aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.344362 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d246a-3a89-42ff-9b5e-ddcfd0f83aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.344996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d246a-3a89-42ff-9b5e-ddcfd0f83aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:34.346577 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d246a-3a89-42ff-9b5e-ddcfd0f83aa3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:34.357298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1233/2453] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:34.362156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79a68a-456b-43d6-b246-44d0f69a7366 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:34.371892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79a68a-456b-43d6-b246-44d0f69a7366 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:34.377585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3bc9b-1df0-4fad-bb33-af5e74864e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:34.378412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3bc9b-1df0-4fad-bb33-af5e74864e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.378699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3bc9b-1df0-4fad-bb33-af5e74864e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.378968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3bc9b-1df0-4fad-bb33-af5e74864e85 admin admin] PATH_INFO: `/v3/projects/98effe0569824c2a859b162b7bcbf31f/users/0ecd8342407d4660aec5632d585dc7b2/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.388243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:34.396382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3bc9b-1df0-4fad-bb33-af5e74864e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98effe0569824c2a859b162b7bcbf31f, user_id=0ecd8342407d4660aec5632d585dc7b2, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:34.399370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3bc9b-1df0-4fad-bb33-af5e74864e85 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:34.412516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1221/2454] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:34 2022] PUT /identity/v3/projects/98effe0569824c2a859b162b7bcbf31f/users/0ecd8342407d4660aec5632d585dc7b2/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:34.423730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289925e-2ad6-4b6d-b205-a9f54ec3e9e0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.424012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289925e-2ad6-4b6d-b205-a9f54ec3e9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.424279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289925e-2ad6-4b6d-b205-a9f54ec3e9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.452736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6289925e-2ad6-4b6d-b205-a9f54ec3e9e0 None None] MFA Rules not processed for user `0ecd8342407d4660aec5632d585dc7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:34.453947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6289925e-2ad6-4b6d-b205-a9f54ec3e9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:34.518156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1234/2455] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:34.522685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3bc9b-1df0-4fad-bb33-af5e74864e85 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:34.583928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3bc9b-1df0-4fad-bb33-af5e74864e85 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:34.588581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd2947-78f8-4575-9f6b-55a15d6f83cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:34.589188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd2947-78f8-4575-9f6b-55a15d6f83cd admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.589467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd2947-78f8-4575-9f6b-55a15d6f83cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.589740 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd2947-78f8-4575-9f6b-55a15d6f83cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.590398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd2947-78f8-4575-9f6b-55a15d6f83cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:34.592364 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd2947-78f8-4575-9f6b-55a15d6f83cd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:34.621084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1222/2456] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:34 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:34.625995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289925e-2ad6-4b6d-b205-a9f54ec3e9e0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:34.636641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289925e-2ad6-4b6d-b205-a9f54ec3e9e0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:34.648805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:34.699660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07620a-6578-4307-89d4-577cc6d0b572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:34.700415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07620a-6578-4307-89d4-577cc6d0b572 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.700697 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07620a-6578-4307-89d4-577cc6d0b572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.700966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07620a-6578-4307-89d4-577cc6d0b572 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.701653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07620a-6578-4307-89d4-577cc6d0b572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:34.704315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07620a-6578-4307-89d4-577cc6d0b572 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:34.731370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1235/2457] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:34 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:34.732550 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd2947-78f8-4575-9f6b-55a15d6f83cd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:34.744222 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd2947-78f8-4575-9f6b-55a15d6f83cd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:34.749434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf53f6e-df35-4b76-9ae6-f4689cd61dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:34.750081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf53f6e-df35-4b76-9ae6-f4689cd61dfd admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.750485 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf53f6e-df35-4b76-9ae6-f4689cd61dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.750771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf53f6e-df35-4b76-9ae6-f4689cd61dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.751451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf53f6e-df35-4b76-9ae6-f4689cd61dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:34.753059 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf53f6e-df35-4b76-9ae6-f4689cd61dfd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:34.764468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1223/2458] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:34.768995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07620a-6578-4307-89d4-577cc6d0b572 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:34.779131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07620a-6578-4307-89d4-577cc6d0b572 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:34.784955 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460c077-2ba6-435e-a849-cdaf615d1042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:34.785671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460c077-2ba6-435e-a849-cdaf615d1042 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.785956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460c077-2ba6-435e-a849-cdaf615d1042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.786234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460c077-2ba6-435e-a849-cdaf615d1042 admin admin] PATH_INFO: `/v3/projects/a01277d9db394f56a1b9396257792abb/users/eba63fe80d83439c8ae269fc68f60068/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.802445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460c077-2ba6-435e-a849-cdaf615d1042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a01277d9db394f56a1b9396257792abb, user_id=eba63fe80d83439c8ae269fc68f60068, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:34.805220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460c077-2ba6-435e-a849-cdaf615d1042 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:34.823079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1236/2459] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:34 2022] PUT /identity/v3/projects/a01277d9db394f56a1b9396257792abb/users/eba63fe80d83439c8ae269fc68f60068/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:34.827574 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf53f6e-df35-4b76-9ae6-f4689cd61dfd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:34.837591 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf53f6e-df35-4b76-9ae6-f4689cd61dfd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:34.895211 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872bcaa-bc6d-4065-82d1-97e0dd3136d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:34.895825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872bcaa-bc6d-4065-82d1-97e0dd3136d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.896101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872bcaa-bc6d-4065-82d1-97e0dd3136d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.896371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872bcaa-bc6d-4065-82d1-97e0dd3136d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.897008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872bcaa-bc6d-4065-82d1-97e0dd3136d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:34.898593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872bcaa-bc6d-4065-82d1-97e0dd3136d2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:34.908886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1224/2460] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:34.913074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460c077-2ba6-435e-a849-cdaf615d1042 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:34.923916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460c077-2ba6-435e-a849-cdaf615d1042 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:34.928606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9453c-04d7-409d-b62a-51d89893e23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:34.929248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9453c-04d7-409d-b62a-51d89893e23c admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.929524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9453c-04d7-409d-b62a-51d89893e23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.929797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9453c-04d7-409d-b62a-51d89893e23c admin admin] PATH_INFO: `/v3/projects/a01277d9db394f56a1b9396257792abb/users/eba63fe80d83439c8ae269fc68f60068/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.931730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9453c-04d7-409d-b62a-51d89893e23c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a01277d9db394f56a1b9396257792abb, user_id=eba63fe80d83439c8ae269fc68f60068, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:34.934357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9453c-04d7-409d-b62a-51d89893e23c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:34.945291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1237/2461] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:34 2022] PUT /identity/v3/projects/a01277d9db394f56a1b9396257792abb/users/eba63fe80d83439c8ae269fc68f60068/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:34.956583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b4bd1-06c6-48d7-98e6-cdeda434d4aa None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:34.956871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b4bd1-06c6-48d7-98e6-cdeda434d4aa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:34.957145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b4bd1-06c6-48d7-98e6-cdeda434d4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:34.986410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ea8b4bd1-06c6-48d7-98e6-cdeda434d4aa None None] MFA Rules not processed for user `eba63fe80d83439c8ae269fc68f60068`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:34.987697 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ea8b4bd1-06c6-48d7-98e6-cdeda434d4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:35.037487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:35.065533 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1225/2462] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:35.077513 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9453c-04d7-409d-b62a-51d89893e23c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:35.079458 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aaa9453c-04d7-409d-b62a-51d89893e23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:35.138700 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9453c-04d7-409d-b62a-51d89893e23c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:35.143027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5da68e-9068-42e4-8eb0-a0e7ae3ee475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:35.143655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5da68e-9068-42e4-8eb0-a0e7ae3ee475 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:35.143938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5da68e-9068-42e4-8eb0-a0e7ae3ee475 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:35.144212 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5da68e-9068-42e4-8eb0-a0e7ae3ee475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:35.153203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5da68e-9068-42e4-8eb0-a0e7ae3ee475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:35.154671 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5da68e-9068-42e4-8eb0-a0e7ae3ee475 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:35.164622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1238/2463] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:35 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:35.230503 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b4bd1-06c6-48d7-98e6-cdeda434d4aa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:35.296981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b4bd1-06c6-48d7-98e6-cdeda434d4aa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:35.301924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:35.303001 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:35.303315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:35.303589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] PATH_INFO: `/v3/users/0ecd8342407d4660aec5632d585dc7b2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:35.304728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ecd8342407d4660aec5632d585dc7b2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:35.306663 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:35.358819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ecd8342407d4660aec5632d585dc7b2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:35.367027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ecd8342407d4660aec5632d585dc7b2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:35.382454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ecd8342407d4660aec5632d585dc7b2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:35.383307 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ecd8342407d4660aec5632d585dc7b2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:35.394351 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ecd8342407d4660aec5632d585dc7b2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:35.400940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1226/2464] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:35 2022] DELETE /identity/v3/users/0ecd8342407d4660aec5632d585dc7b2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:35.406066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298d296-b409-4af7-a22f-d355aa9604dc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:35.406356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298d296-b409-4af7-a22f-d355aa9604dc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:35.406644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298d296-b409-4af7-a22f-d355aa9604dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:35.427540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e298d296-b409-4af7-a22f-d355aa9604dc None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:35.428759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e298d296-b409-4af7-a22f-d355aa9604dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:35.492630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1239/2465] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:35.503090 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:35.504543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:35.572474 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9651ff-7eb4-4c43-928e-817702751808 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:35.576581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cedfad-0d10-4dc7-a364-680be50f288c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:35.577522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cedfad-0d10-4dc7-a364-680be50f288c service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:35.577972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cedfad-0d10-4dc7-a364-680be50f288c service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:35.578256 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cedfad-0d10-4dc7-a364-680be50f288c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:35.588374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cedfad-0d10-4dc7-a364-680be50f288c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:35.589946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cedfad-0d10-4dc7-a364-680be50f288c service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:35.603505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1227/2466] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:36.284904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298d296-b409-4af7-a22f-d355aa9604dc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:36.286536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e298d296-b409-4af7-a22f-d355aa9604dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:36.298838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298d296-b409-4af7-a22f-d355aa9604dc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:36.303809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:36.304542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:36.304952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:36.305295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] PATH_INFO: `/v3/projects/98effe0569824c2a859b162b7bcbf31f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:36.306504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98effe0569824c2a859b162b7bcbf31f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:36.308666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:36.319147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:36.401320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98effe0569824c2a859b162b7bcbf31f', 'request_id': 'req-60b38fc7-9c15-46ed-964f-0ff54a805cd9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:36.410839 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98effe0569824c2a859b162b7bcbf31f', 'request_id': 'req-60b38fc7-9c15-46ed-964f-0ff54a805cd9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:36.419405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98effe0569824c2a859b162b7bcbf31f', 'request_id': 'req-60b38fc7-9c15-46ed-964f-0ff54a805cd9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:36.440355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] The token cache is being invalidate because project 98effe0569824c2a859b162b7bcbf31f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:36.440654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:36.441817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:36.444207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1240/2467] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:36 2022] DELETE /identity/v3/projects/98effe0569824c2a859b162b7bcbf31f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:36.448482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cedfad-0d10-4dc7-a364-680be50f288c service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:36.455753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:36.460180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cedfad-0d10-4dc7-a364-680be50f288c service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:36.560641 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9180d-ebff-4595-8865-37f951149a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:36.562386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9180d-ebff-4595-8865-37f951149a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:36.563481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9180d-ebff-4595-8865-37f951149a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:36.564067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9180d-ebff-4595-8865-37f951149a88 admin admin] PATH_INFO: `/v3/users/eba63fe80d83439c8ae269fc68f60068` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:36.565560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9180d-ebff-4595-8865-37f951149a88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eba63fe80d83439c8ae269fc68f60068)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:36.568114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9180d-ebff-4595-8865-37f951149a88 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:36.656634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13d9180d-ebff-4595-8865-37f951149a88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eba63fe80d83439c8ae269fc68f60068'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:36.667013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13d9180d-ebff-4595-8865-37f951149a88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eba63fe80d83439c8ae269fc68f60068'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:36.685428 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13d9180d-ebff-4595-8865-37f951149a88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eba63fe80d83439c8ae269fc68f60068'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:36.687701 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13d9180d-ebff-4595-8865-37f951149a88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eba63fe80d83439c8ae269fc68f60068'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:36.696944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13d9180d-ebff-4595-8865-37f951149a88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eba63fe80d83439c8ae269fc68f60068'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:36.708325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1228/2468] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:36 2022] DELETE /identity/v3/users/eba63fe80d83439c8ae269fc68f60068 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:37.018405 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:37.020244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:37.051942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b38fc7-9c15-46ed-964f-0ff54a805cd9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:37.114890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148d58c-fc47-4202-abfe-d0f9dbe725d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:37.115646 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148d58c-fc47-4202-abfe-d0f9dbe725d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:37.115928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148d58c-fc47-4202-abfe-d0f9dbe725d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:37.116196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148d58c-fc47-4202-abfe-d0f9dbe725d3 admin admin] PATH_INFO: `/v3/projects/a01277d9db394f56a1b9396257792abb` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:37.117315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148d58c-fc47-4202-abfe-d0f9dbe725d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a01277d9db394f56a1b9396257792abb)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:37.119437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148d58c-fc47-4202-abfe-d0f9dbe725d3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:37.170350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d148d58c-fc47-4202-abfe-d0f9dbe725d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a01277d9db394f56a1b9396257792abb', 'request_id': 'req-d148d58c-fc47-4202-abfe-d0f9dbe725d3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:37.179235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d148d58c-fc47-4202-abfe-d0f9dbe725d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a01277d9db394f56a1b9396257792abb', 'request_id': 'req-d148d58c-fc47-4202-abfe-d0f9dbe725d3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:37.184869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d148d58c-fc47-4202-abfe-d0f9dbe725d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a01277d9db394f56a1b9396257792abb', 'request_id': 'req-d148d58c-fc47-4202-abfe-d0f9dbe725d3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:37.196672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d148d58c-fc47-4202-abfe-d0f9dbe725d3 admin admin] The token cache is being invalidate because project a01277d9db394f56a1b9396257792abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:37.196958 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d148d58c-fc47-4202-abfe-d0f9dbe725d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:37.197590 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d148d58c-fc47-4202-abfe-d0f9dbe725d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:37.199141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1241/2469] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:37 2022] DELETE /identity/v3/projects/a01277d9db394f56a1b9396257792abb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:37.204278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a5410-27da-44c0-a249-fe1f83c8f279 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:37.204658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a5410-27da-44c0-a249-fe1f83c8f279 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:37.204939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a5410-27da-44c0-a249-fe1f83c8f279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:37.226430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b94a5410-27da-44c0-a249-fe1f83c8f279 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:37.227766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b94a5410-27da-44c0-a249-fe1f83c8f279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:37.294987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1229/2470] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:37.303853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f937d-698a-44ce-93d9-926efc88dcfd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:37.304318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f937d-698a-44ce-93d9-926efc88dcfd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:37.304436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f937d-698a-44ce-93d9-926efc88dcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:37.324243 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-974f937d-698a-44ce-93d9-926efc88dcfd None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:37.325447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-974f937d-698a-44ce-93d9-926efc88dcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:37.334282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1242/2471] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:37.338658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a5410-27da-44c0-a249-fe1f83c8f279 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:37.349034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a5410-27da-44c0-a249-fe1f83c8f279 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:37.353882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eadd75-3aab-4420-ba59-186a3b3bf015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:37.354483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eadd75-3aab-4420-ba59-186a3b3bf015 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:37.354760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eadd75-3aab-4420-ba59-186a3b3bf015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:37.355031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eadd75-3aab-4420-ba59-186a3b3bf015 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:37.355729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eadd75-3aab-4420-ba59-186a3b3bf015 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:21:37.356033 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eadd75-3aab-4420-ba59-186a3b3bf015 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:37.357687 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eadd75-3aab-4420-ba59-186a3b3bf015 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:37.369381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1230/2472] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:21:37.374266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f937d-698a-44ce-93d9-926efc88dcfd None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:37.383517 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f937d-698a-44ce-93d9-926efc88dcfd None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:37.388123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7bf55-5b3e-44c2-8aa5-6699a53110da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:37.388714 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7bf55-5b3e-44c2-8aa5-6699a53110da admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:37.389509 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7bf55-5b3e-44c2-8aa5-6699a53110da admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:37.389924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7bf55-5b3e-44c2-8aa5-6699a53110da admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:37.390732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7bf55-5b3e-44c2-8aa5-6699a53110da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:37.392672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7bf55-5b3e-44c2-8aa5-6699a53110da admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:37.414078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:37.422327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1243/2473] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:37 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:37.426494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eadd75-3aab-4420-ba59-186a3b3bf015 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:37.436471 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eadd75-3aab-4420-ba59-186a3b3bf015 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:37.445714 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:37.495036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757854d-7dcd-491c-82f8-9a220ce4aed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:37.495769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757854d-7dcd-491c-82f8-9a220ce4aed3 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:37.495945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757854d-7dcd-491c-82f8-9a220ce4aed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:37.496241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757854d-7dcd-491c-82f8-9a220ce4aed3 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:37.496874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757854d-7dcd-491c-82f8-9a220ce4aed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:37.499144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757854d-7dcd-491c-82f8-9a220ce4aed3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:37.521312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1231/2474] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:37 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:37.525889 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7bf55-5b3e-44c2-8aa5-6699a53110da admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:37.536860 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7bf55-5b3e-44c2-8aa5-6699a53110da admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:37.541496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152ead1-27ae-494e-b07f-fdab67bab4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:37.542089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152ead1-27ae-494e-b07f-fdab67bab4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:37.542367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152ead1-27ae-494e-b07f-fdab67bab4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:37.542638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152ead1-27ae-494e-b07f-fdab67bab4d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:37.543310 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152ead1-27ae-494e-b07f-fdab67bab4d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:37.544891 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152ead1-27ae-494e-b07f-fdab67bab4d2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:37.554908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1244/2475] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:37.558961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757854d-7dcd-491c-82f8-9a220ce4aed3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:37.569053 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757854d-7dcd-491c-82f8-9a220ce4aed3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:37.573899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65427c8-49d0-42b4-9af0-234c2921b29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:37.574502 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65427c8-49d0-42b4-9af0-234c2921b29f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:37.574786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65427c8-49d0-42b4-9af0-234c2921b29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:37.575075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65427c8-49d0-42b4-9af0-234c2921b29f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:37.575736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65427c8-49d0-42b4-9af0-234c2921b29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:37.578027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65427c8-49d0-42b4-9af0-234c2921b29f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:37.588735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1232/2476] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:37.593924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152ead1-27ae-494e-b07f-fdab67bab4d2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:37.604161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152ead1-27ae-494e-b07f-fdab67bab4d2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:37.609157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ed1e8-0076-41e6-bdd7-9de3676517dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:37.609810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed1e8-0076-41e6-bdd7-9de3676517dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:37.610091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed1e8-0076-41e6-bdd7-9de3676517dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:37.610837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed1e8-0076-41e6-bdd7-9de3676517dd admin admin] PATH_INFO: `/v3/projects/251a6b9f3a9840729409f29297719a97/users/bf61b36f760a44eeaec33f8844d20bed/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:37.629075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ed1e8-0076-41e6-bdd7-9de3676517dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=251a6b9f3a9840729409f29297719a97, user_id=bf61b36f760a44eeaec33f8844d20bed, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:37.631861 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ed1e8-0076-41e6-bdd7-9de3676517dd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:37.647880 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1245/2477] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:37 2022] PUT /identity/v3/projects/251a6b9f3a9840729409f29297719a97/users/bf61b36f760a44eeaec33f8844d20bed/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:21:37.660044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7908a5-cb8f-4f59-8f2b-d54ae60f9025 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:37.660501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7908a5-cb8f-4f59-8f2b-d54ae60f9025 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:37.660915 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7908a5-cb8f-4f59-8f2b-d54ae60f9025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:37.690345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3a7908a5-cb8f-4f59-8f2b-d54ae60f9025 None None] MFA Rules not processed for user `bf61b36f760a44eeaec33f8844d20bed`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:37.691721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3a7908a5-cb8f-4f59-8f2b-d54ae60f9025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:37.761485 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1233/2478] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:37.766968 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ed1e8-0076-41e6-bdd7-9de3676517dd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:37.834596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ed1e8-0076-41e6-bdd7-9de3676517dd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:37.839299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76202511-7bdb-4b8d-ab72-5636cb819b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:37.839903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76202511-7bdb-4b8d-ab72-5636cb819b64 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:37.840182 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76202511-7bdb-4b8d-ab72-5636cb819b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:37.840453 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76202511-7bdb-4b8d-ab72-5636cb819b64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:37.841092 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76202511-7bdb-4b8d-ab72-5636cb819b64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:37.843030 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76202511-7bdb-4b8d-ab72-5636cb819b64 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:37.869425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1246/2479] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:37 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:37.874477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7908a5-cb8f-4f59-8f2b-d54ae60f9025 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:37.885926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7908a5-cb8f-4f59-8f2b-d54ae60f9025 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:37.942528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c329fd-9f06-40d3-bca7-1b0c7037fa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:37.943073 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c329fd-9f06-40d3-bca7-1b0c7037fa89 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:37.943354 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c329fd-9f06-40d3-bca7-1b0c7037fa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:37.943648 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c329fd-9f06-40d3-bca7-1b0c7037fa89 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:37.944274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c329fd-9f06-40d3-bca7-1b0c7037fa89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:37.946185 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c329fd-9f06-40d3-bca7-1b0c7037fa89 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:37.968357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1234/2480] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:37 2022] POST /identity/v3/users => generated 301 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:37.972454 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76202511-7bdb-4b8d-ab72-5636cb819b64 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:37.983035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76202511-7bdb-4b8d-ab72-5636cb819b64 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:37.987809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12c574-d9bb-4542-a47b-b591fbe8ac0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:37.988596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12c574-d9bb-4542-a47b-b591fbe8ac0a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:37.988879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12c574-d9bb-4542-a47b-b591fbe8ac0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:37.989147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12c574-d9bb-4542-a47b-b591fbe8ac0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:37.989793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12c574-d9bb-4542-a47b-b591fbe8ac0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:37.991426 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12c574-d9bb-4542-a47b-b591fbe8ac0a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:38.003237 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1247/2481] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:21:38.007367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c329fd-9f06-40d3-bca7-1b0c7037fa89 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:38.018117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c329fd-9f06-40d3-bca7-1b0c7037fa89 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:38.022798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18b34c-2d25-4c26-8524-85e3c2f36868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:38.023487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18b34c-2d25-4c26-8524-85e3c2f36868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:38.023768 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18b34c-2d25-4c26-8524-85e3c2f36868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:38.024041 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18b34c-2d25-4c26-8524-85e3c2f36868 admin admin] PATH_INFO: `/v3/projects/1c74994779e94eff87926161944254f5/users/b9e41eb899204293988b4c07469bed3f/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:38.027112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:38.039904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18b34c-2d25-4c26-8524-85e3c2f36868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c74994779e94eff87926161944254f5, user_id=b9e41eb899204293988b4c07469bed3f, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:38.042547 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18b34c-2d25-4c26-8524-85e3c2f36868 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:38.055230 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1235/2482] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:38 2022] PUT /identity/v3/projects/1c74994779e94eff87926161944254f5/users/b9e41eb899204293988b4c07469bed3f/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:21:38.059203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12c574-d9bb-4542-a47b-b591fbe8ac0a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:38.069910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12c574-d9bb-4542-a47b-b591fbe8ac0a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:38.131597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61ce53-6288-41e4-baa2-0b7f45361a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:38.132209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61ce53-6288-41e4-baa2-0b7f45361a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:38.132490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61ce53-6288-41e4-baa2-0b7f45361a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:38.132767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61ce53-6288-41e4-baa2-0b7f45361a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:38.133407 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61ce53-6288-41e4-baa2-0b7f45361a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:38.135020 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61ce53-6288-41e4-baa2-0b7f45361a97 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:38.145127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1248/2483] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:38.149381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18b34c-2d25-4c26-8524-85e3c2f36868 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:38.162173 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18b34c-2d25-4c26-8524-85e3c2f36868 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:38.166896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d086d-c3eb-402e-9d7b-591243bb8921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:38.167582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d086d-c3eb-402e-9d7b-591243bb8921 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:38.167859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d086d-c3eb-402e-9d7b-591243bb8921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:38.168127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d086d-c3eb-402e-9d7b-591243bb8921 admin admin] PATH_INFO: `/v3/projects/1c74994779e94eff87926161944254f5/users/b9e41eb899204293988b4c07469bed3f/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:38.170044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d086d-c3eb-402e-9d7b-591243bb8921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c74994779e94eff87926161944254f5, user_id=b9e41eb899204293988b4c07469bed3f, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:38.173363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d086d-c3eb-402e-9d7b-591243bb8921 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:38.187379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1236/2484] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:38 2022] PUT /identity/v3/projects/1c74994779e94eff87926161944254f5/users/b9e41eb899204293988b4c07469bed3f/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:38.261794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b15d8b-0f12-45df-bb6c-e682e990411e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:38.261987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b15d8b-0f12-45df-bb6c-e682e990411e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:38.262259 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b15d8b-0f12-45df-bb6c-e682e990411e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:38.293490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-15b15d8b-0f12-45df-bb6c-e682e990411e None None] MFA Rules not processed for user `b9e41eb899204293988b4c07469bed3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:38.294867 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-15b15d8b-0f12-45df-bb6c-e682e990411e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:38.319985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:38.382915 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1249/2485] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:38 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:38.393903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d086d-c3eb-402e-9d7b-591243bb8921 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:38.395649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-922d086d-c3eb-402e-9d7b-591243bb8921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:38.458785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d086d-c3eb-402e-9d7b-591243bb8921 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:38.463113 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5858fd-c1cf-4e7b-94c2-153b8e85b406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:38.463724 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5858fd-c1cf-4e7b-94c2-153b8e85b406 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:38.464058 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5858fd-c1cf-4e7b-94c2-153b8e85b406 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:38.464408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5858fd-c1cf-4e7b-94c2-153b8e85b406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:38.477716 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5858fd-c1cf-4e7b-94c2-153b8e85b406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:38.479180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5858fd-c1cf-4e7b-94c2-153b8e85b406 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:38.491578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1237/2486] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:38 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:38.662143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b15d8b-0f12-45df-bb6c-e682e990411e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:38.757491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b15d8b-0f12-45df-bb6c-e682e990411e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:38.762814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:38.763751 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:38.764014 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:38.764285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] PATH_INFO: `/v3/users/bf61b36f760a44eeaec33f8844d20bed` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:38.765618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf61b36f760a44eeaec33f8844d20bed)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:38.768172 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:38.829787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf61b36f760a44eeaec33f8844d20bed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:38.837366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf61b36f760a44eeaec33f8844d20bed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:38.838066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf61b36f760a44eeaec33f8844d20bed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:38.855586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf61b36f760a44eeaec33f8844d20bed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:38.864694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf61b36f760a44eeaec33f8844d20bed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:38.870708 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1250/2487] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:38 2022] DELETE /identity/v3/users/bf61b36f760a44eeaec33f8844d20bed => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:38.877167 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021b02-777e-4bb3-a907-6e39acd4b0c0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:38.877686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021b02-777e-4bb3-a907-6e39acd4b0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:38.877686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021b02-777e-4bb3-a907-6e39acd4b0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:38.897736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f7021b02-777e-4bb3-a907-6e39acd4b0c0 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:38.899078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f7021b02-777e-4bb3-a907-6e39acd4b0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:38.966959 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1238/2488] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:38.979218 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:38.982991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:39.037220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:39.055609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b270ba-25d2-4a6d-ab1b-3b2f4cffb462 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:39.059797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7a842-c51f-4a6e-93c4-90e4ed4a7994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:39.060433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a842-c51f-4a6e-93c4-90e4ed4a7994 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:39.060713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a842-c51f-4a6e-93c4-90e4ed4a7994 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:39.061104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a842-c51f-4a6e-93c4-90e4ed4a7994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:39.070755 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7a842-c51f-4a6e-93c4-90e4ed4a7994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:39.072289 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7a842-c51f-4a6e-93c4-90e4ed4a7994 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:39.083489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1251/2489] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:39.579237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf618f9-edf3-4a15-b272-6426f64b99ab None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:39.579237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf618f9-edf3-4a15-b272-6426f64b99ab None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:39.579237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf618f9-edf3-4a15-b272-6426f64b99ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:39.599788 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-caf618f9-edf3-4a15-b272-6426f64b99ab None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:39.601519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-caf618f9-edf3-4a15-b272-6426f64b99ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:39.667432 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1239/2490] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 9 13:21:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:21:39.728338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7a842-c51f-4a6e-93c4-90e4ed4a7994 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:39.729821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3af7a842-c51f-4a6e-93c4-90e4ed4a7994 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:39.812074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7a842-c51f-4a6e-93c4-90e4ed4a7994 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:39.818292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304bf8-cd90-4150-a89f-285ca78e0b6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:39.818927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304bf8-cd90-4150-a89f-285ca78e0b6f service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:39.819255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304bf8-cd90-4150-a89f-285ca78e0b6f service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:39.819526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304bf8-cd90-4150-a89f-285ca78e0b6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:39.830186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf618f9-edf3-4a15-b272-6426f64b99ab None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:39.832176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-caf618f9-edf3-4a15-b272-6426f64b99ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:39.839925 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7304bf8-cd90-4150-a89f-285ca78e0b6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:39.841938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7304bf8-cd90-4150-a89f-285ca78e0b6f service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:39.853698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf618f9-edf3-4a15-b272-6426f64b99ab None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:39.854308 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1252/2491] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:21:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:21:39.861751 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:39.862478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:39.863442 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:39.863840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] PATH_INFO: `/v3/projects/251a6b9f3a9840729409f29297719a97` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:39.866128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=251a6b9f3a9840729409f29297719a97)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:39.869592 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:39.895880 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:39.936274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '251a6b9f3a9840729409f29297719a97', 'request_id': 'req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:39.944258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '251a6b9f3a9840729409f29297719a97', 'request_id': 'req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:39.955380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '251a6b9f3a9840729409f29297719a97', 'request_id': 'req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:39.987215 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] The token cache is being invalidate because project 251a6b9f3a9840729409f29297719a97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:39.987519 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:39.988280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:39.990064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1240/2492] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:39 2022] DELETE /identity/v3/projects/251a6b9f3a9840729409f29297719a97 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:39.994057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304bf8-cd90-4150-a89f-285ca78e0b6f service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:40.007386 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304bf8-cd90-4150-a89f-285ca78e0b6f service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:40.074846 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810da27d-42cf-4db9-b5c8-e6ef46270ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:40.075644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810da27d-42cf-4db9-b5c8-e6ef46270ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:40.076000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810da27d-42cf-4db9-b5c8-e6ef46270ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:40.076345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810da27d-42cf-4db9-b5c8-e6ef46270ab7 admin admin] PATH_INFO: `/v3/users/b9e41eb899204293988b4c07469bed3f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:40.079341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810da27d-42cf-4db9-b5c8-e6ef46270ab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9e41eb899204293988b4c07469bed3f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:40.081975 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810da27d-42cf-4db9-b5c8-e6ef46270ab7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:40.146712 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-810da27d-42cf-4db9-b5c8-e6ef46270ab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9e41eb899204293988b4c07469bed3f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:40.156190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-810da27d-42cf-4db9-b5c8-e6ef46270ab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9e41eb899204293988b4c07469bed3f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:40.156970 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-810da27d-42cf-4db9-b5c8-e6ef46270ab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9e41eb899204293988b4c07469bed3f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:40.172792 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-810da27d-42cf-4db9-b5c8-e6ef46270ab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9e41eb899204293988b4c07469bed3f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:40.174161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:40.182085 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-810da27d-42cf-4db9-b5c8-e6ef46270ab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9e41eb899204293988b4c07469bed3f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:40.191696 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1253/2493] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:39 2022] DELETE /identity/v3/users/b9e41eb899204293988b4c07469bed3f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:40.614717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:40.616646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:40.648094 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39abc18-fbc4-4abf-9312-8f9702b1ed1d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:40.763726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5276d7-e056-4da5-982b-6171a1712ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:40.765895 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5276d7-e056-4da5-982b-6171a1712ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:40.766417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5276d7-e056-4da5-982b-6171a1712ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:40.766904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5276d7-e056-4da5-982b-6171a1712ea6 admin admin] PATH_INFO: `/v3/projects/1c74994779e94eff87926161944254f5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:40.768315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5276d7-e056-4da5-982b-6171a1712ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c74994779e94eff87926161944254f5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:40.770886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5276d7-e056-4da5-982b-6171a1712ea6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:40.829352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cc5276d7-e056-4da5-982b-6171a1712ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c74994779e94eff87926161944254f5', 'request_id': 'req-cc5276d7-e056-4da5-982b-6171a1712ea6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:40.836044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cc5276d7-e056-4da5-982b-6171a1712ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c74994779e94eff87926161944254f5', 'request_id': 'req-cc5276d7-e056-4da5-982b-6171a1712ea6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:40.844234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cc5276d7-e056-4da5-982b-6171a1712ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c74994779e94eff87926161944254f5', 'request_id': 'req-cc5276d7-e056-4da5-982b-6171a1712ea6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:40.860426 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cc5276d7-e056-4da5-982b-6171a1712ea6 admin admin] The token cache is being invalidate because project 1c74994779e94eff87926161944254f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:40.860826 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cc5276d7-e056-4da5-982b-6171a1712ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:40.861537 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cc5276d7-e056-4da5-982b-6171a1712ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:40.863398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1241/2494] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:40 2022] DELETE /identity/v3/projects/1c74994779e94eff87926161944254f5 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:40.868062 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19c396-0884-4c17-af67-880248af6560 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:40.868516 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19c396-0884-4c17-af67-880248af6560 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:40.868629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19c396-0884-4c17-af67-880248af6560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:40.889775 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ab19c396-0884-4c17-af67-880248af6560 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:40.891007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ab19c396-0884-4c17-af67-880248af6560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:40.964295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1254/2495] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:40.973038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b3ee0-032f-431e-abc9-e4691b02d339 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:40.974278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b3ee0-032f-431e-abc9-e4691b02d339 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:40.974763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b3ee0-032f-431e-abc9-e4691b02d339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:41.004532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-458b3ee0-032f-431e-abc9-e4691b02d339 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:41.006228 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-458b3ee0-032f-431e-abc9-e4691b02d339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:41.016251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1242/2496] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:41.020945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19c396-0884-4c17-af67-880248af6560 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:41.033099 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19c396-0884-4c17-af67-880248af6560 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:41.044990 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ad403-1ce5-4709-abee-48dcd0f34258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:41.046224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ad403-1ce5-4709-abee-48dcd0f34258 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:41.046520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ad403-1ce5-4709-abee-48dcd0f34258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:41.046797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ad403-1ce5-4709-abee-48dcd0f34258 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:41.048219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ad403-1ce5-4709-abee-48dcd0f34258 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:21:41.048537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ad403-1ce5-4709-abee-48dcd0f34258 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:41.050355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ad403-1ce5-4709-abee-48dcd0f34258 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:41.067090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1255/2497] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:21:41.069926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b3ee0-032f-431e-abc9-e4691b02d339 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:41.081213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b3ee0-032f-431e-abc9-e4691b02d339 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:41.089612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e81c8-4619-4d5f-9f17-3c47978976f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:41.090247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e81c8-4619-4d5f-9f17-3c47978976f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:41.090610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e81c8-4619-4d5f-9f17-3c47978976f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:41.090852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e81c8-4619-4d5f-9f17-3c47978976f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:41.091541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e81c8-4619-4d5f-9f17-3c47978976f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:41.095527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e81c8-4619-4d5f-9f17-3c47978976f3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:41.141412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1243/2498] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:41 2022] POST /identity/v3/projects => generated 361 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:41.142515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ad403-1ce5-4709-abee-48dcd0f34258 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:41.154094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ad403-1ce5-4709-abee-48dcd0f34258 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:41.227606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0d78c-6b8b-41d5-a6d6-28fd3d1a898c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:41.227873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0d78c-6b8b-41d5-a6d6-28fd3d1a898c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:41.228230 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0d78c-6b8b-41d5-a6d6-28fd3d1a898c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:41.228580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0d78c-6b8b-41d5-a6d6-28fd3d1a898c admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:41.229303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0d78c-6b8b-41d5-a6d6-28fd3d1a898c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:41.231447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0d78c-6b8b-41d5-a6d6-28fd3d1a898c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:41.257161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1256/2499] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:41 2022] POST /identity/v3/users => generated 281 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:21:41.262349 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e81c8-4619-4d5f-9f17-3c47978976f3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:41.274017 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:41.279826 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e81c8-4619-4d5f-9f17-3c47978976f3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:41.291498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed29004-eeec-43d4-ac9e-c963c275d19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:41.292186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29004-eeec-43d4-ac9e-c963c275d19b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:41.292565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29004-eeec-43d4-ac9e-c963c275d19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:41.292907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29004-eeec-43d4-ac9e-c963c275d19b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:41.293636 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed29004-eeec-43d4-ac9e-c963c275d19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:41.295936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed29004-eeec-43d4-ac9e-c963c275d19b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:41.308288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1244/2500] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:41.312860 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0d78c-6b8b-41d5-a6d6-28fd3d1a898c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:41.326105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0d78c-6b8b-41d5-a6d6-28fd3d1a898c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:41.331903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec1de9-0025-4c2c-bd80-a393b72eaaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:41.332529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec1de9-0025-4c2c-bd80-a393b72eaaa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:41.333190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec1de9-0025-4c2c-bd80-a393b72eaaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:41.333190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec1de9-0025-4c2c-bd80-a393b72eaaa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:41.333721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec1de9-0025-4c2c-bd80-a393b72eaaa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:41.335360 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec1de9-0025-4c2c-bd80-a393b72eaaa4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:41.354416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1257/2501] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:41.359701 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed29004-eeec-43d4-ac9e-c963c275d19b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:41.370187 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed29004-eeec-43d4-ac9e-c963c275d19b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:41.376520 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128004c-07ba-4eb8-9577-6b5acfa2b0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:41.376968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128004c-07ba-4eb8-9577-6b5acfa2b0e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:41.377246 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128004c-07ba-4eb8-9577-6b5acfa2b0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:41.377522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128004c-07ba-4eb8-9577-6b5acfa2b0e5 admin admin] PATH_INFO: `/v3/projects/49307e25874548b5bb9f42f82e949ba1/users/6271c5e31d394356839494506baed5be/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:41.404658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f128004c-07ba-4eb8-9577-6b5acfa2b0e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49307e25874548b5bb9f42f82e949ba1, user_id=6271c5e31d394356839494506baed5be, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:41.407378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f128004c-07ba-4eb8-9577-6b5acfa2b0e5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:41.427381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1245/2502] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:41 2022] PUT /identity/v3/projects/49307e25874548b5bb9f42f82e949ba1/users/6271c5e31d394356839494506baed5be/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:41.440069 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588a041-8585-4de2-8179-e91bf0e4a701 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:41.440432 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588a041-8585-4de2-8179-e91bf0e4a701 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:41.440615 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588a041-8585-4de2-8179-e91bf0e4a701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:41.475934 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f588a041-8585-4de2-8179-e91bf0e4a701 None None] MFA Rules not processed for user `6271c5e31d394356839494506baed5be`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:41.477529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f588a041-8585-4de2-8179-e91bf0e4a701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:41.542162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:41.574002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1258/2503] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:41.616171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128004c-07ba-4eb8-9577-6b5acfa2b0e5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:41.618176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f128004c-07ba-4eb8-9577-6b5acfa2b0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:41.683516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128004c-07ba-4eb8-9577-6b5acfa2b0e5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:41.687710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44c1e4-67de-4a21-ba6d-6eeca2099669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:41.688364 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44c1e4-67de-4a21-ba6d-6eeca2099669 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:41.688626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44c1e4-67de-4a21-ba6d-6eeca2099669 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:41.688898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44c1e4-67de-4a21-ba6d-6eeca2099669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:41.698317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44c1e4-67de-4a21-ba6d-6eeca2099669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:41.699827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44c1e4-67de-4a21-ba6d-6eeca2099669 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:41.710635 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1246/2504] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:41.727474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588a041-8585-4de2-8179-e91bf0e4a701 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:41.816361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588a041-8585-4de2-8179-e91bf0e4a701 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:41.822573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:41.822573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:41.824039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:41.824039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] PATH_INFO: `/v3/users/6271c5e31d394356839494506baed5be` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:41.824904 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6271c5e31d394356839494506baed5be)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:41.827311 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:41.895459 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6271c5e31d394356839494506baed5be'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:41.910590 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6271c5e31d394356839494506baed5be'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:41.911466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6271c5e31d394356839494506baed5be'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:41.927434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6271c5e31d394356839494506baed5be'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:41.936195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6271c5e31d394356839494506baed5be'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:41.942637 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1259/2505] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:41 2022] DELETE /identity/v3/users/6271c5e31d394356839494506baed5be => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:41.947175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a46b9b-8985-49c4-8e75-e3e19710d491 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:41.947466 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a46b9b-8985-49c4-8e75-e3e19710d491 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:41.947747 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a46b9b-8985-49c4-8e75-e3e19710d491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:41.969218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b1a46b9b-8985-49c4-8e75-e3e19710d491 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:41.970461 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b1a46b9b-8985-49c4-8e75-e3e19710d491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:42.052348 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1247/2506] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:42.068004 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:42.069468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:42.134627 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1f298-d8e9-419e-a76d-9a7f3510c3dd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:42.138307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612cdd2-6ea5-447b-8b07-43d2d410be9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:42.139449 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612cdd2-6ea5-447b-8b07-43d2d410be9b service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:42.139737 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612cdd2-6ea5-447b-8b07-43d2d410be9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:42.140003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612cdd2-6ea5-447b-8b07-43d2d410be9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:42.149678 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612cdd2-6ea5-447b-8b07-43d2d410be9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:42.151205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612cdd2-6ea5-447b-8b07-43d2d410be9b service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:42.162159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1260/2507] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:42.595360 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a46b9b-8985-49c4-8e75-e3e19710d491 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:42.595913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b1a46b9b-8985-49c4-8e75-e3e19710d491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:42.611458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a46b9b-8985-49c4-8e75-e3e19710d491 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:42.616561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4ba27-7528-446f-809e-e52e9591ff65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:42.617226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4ba27-7528-446f-809e-e52e9591ff65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:42.617510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4ba27-7528-446f-809e-e52e9591ff65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:42.617790 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4ba27-7528-446f-809e-e52e9591ff65 admin admin] PATH_INFO: `/v3/projects/49307e25874548b5bb9f42f82e949ba1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:42.618981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4ba27-7528-446f-809e-e52e9591ff65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49307e25874548b5bb9f42f82e949ba1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:42.620984 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4ba27-7528-446f-809e-e52e9591ff65 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:42.717103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5e4ba27-7528-446f-809e-e52e9591ff65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49307e25874548b5bb9f42f82e949ba1', 'request_id': 'req-e5e4ba27-7528-446f-809e-e52e9591ff65'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:42.725940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5e4ba27-7528-446f-809e-e52e9591ff65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49307e25874548b5bb9f42f82e949ba1', 'request_id': 'req-e5e4ba27-7528-446f-809e-e52e9591ff65'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:42.735892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5e4ba27-7528-446f-809e-e52e9591ff65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49307e25874548b5bb9f42f82e949ba1', 'request_id': 'req-e5e4ba27-7528-446f-809e-e52e9591ff65'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:42.740534 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:42.768984 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5e4ba27-7528-446f-809e-e52e9591ff65 admin admin] The token cache is being invalidate because project 49307e25874548b5bb9f42f82e949ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:42.769654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5e4ba27-7528-446f-809e-e52e9591ff65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:42.771010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e5e4ba27-7528-446f-809e-e52e9591ff65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:42.774921 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1248/2508] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:42 2022] DELETE /identity/v3/projects/49307e25874548b5bb9f42f82e949ba1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:42.784184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed2672-4c5f-4223-ada7-b1d8bd4b26b9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:42.784835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed2672-4c5f-4223-ada7-b1d8bd4b26b9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:42.785191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed2672-4c5f-4223-ada7-b1d8bd4b26b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:42.808469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:42.819457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-cbed2672-4c5f-4223-ada7-b1d8bd4b26b9 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:42.821555 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cbed2672-4c5f-4223-ada7-b1d8bd4b26b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:42.910898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1261/2509] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:42.911803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c177767-48ea-4d92-a796-2897b6b54d7a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:42.912632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c177767-48ea-4d92-a796-2897b6b54d7a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:42.912912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c177767-48ea-4d92-a796-2897b6b54d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:42.936606 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3c177767-48ea-4d92-a796-2897b6b54d7a None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:42.938086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3c177767-48ea-4d92-a796-2897b6b54d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:42.947588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1249/2510] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:21:42.951995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed2672-4c5f-4223-ada7-b1d8bd4b26b9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:42.962945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed2672-4c5f-4223-ada7-b1d8bd4b26b9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:42.968809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9858c-2911-4f94-8eb2-697029a40976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:42.969639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9858c-2911-4f94-8eb2-697029a40976 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:42.969918 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9858c-2911-4f94-8eb2-697029a40976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:42.970345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9858c-2911-4f94-8eb2-697029a40976 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:42.971196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9858c-2911-4f94-8eb2-697029a40976 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:21:42.971506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9858c-2911-4f94-8eb2-697029a40976 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:42.973107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9858c-2911-4f94-8eb2-697029a40976 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:42.987334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1262/2511] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:21:42.992093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c177767-48ea-4d92-a796-2897b6b54d7a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:43.002347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c177767-48ea-4d92-a796-2897b6b54d7a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:43.008513 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702da0fe-1a64-49cd-a945-21c2ceb9ff1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:43.009140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702da0fe-1a64-49cd-a945-21c2ceb9ff1b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:43.009520 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702da0fe-1a64-49cd-a945-21c2ceb9ff1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:43.009833 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702da0fe-1a64-49cd-a945-21c2ceb9ff1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:43.010518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702da0fe-1a64-49cd-a945-21c2ceb9ff1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:43.012599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702da0fe-1a64-49cd-a945-21c2ceb9ff1b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:43.050175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1250/2512] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:42 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:21:43.054394 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9858c-2911-4f94-8eb2-697029a40976 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:43.066898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9858c-2911-4f94-8eb2-697029a40976 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:43.146190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a879ac1-2940-425e-8ace-0cd2ff9cdd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:43.146815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a879ac1-2940-425e-8ace-0cd2ff9cdd84 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:43.147121 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a879ac1-2940-425e-8ace-0cd2ff9cdd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:43.147399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a879ac1-2940-425e-8ace-0cd2ff9cdd84 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:43.148057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a879ac1-2940-425e-8ace-0cd2ff9cdd84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:43.150006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a879ac1-2940-425e-8ace-0cd2ff9cdd84 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:43.178806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1263/2513] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:43 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:21:43.180204 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702da0fe-1a64-49cd-a945-21c2ceb9ff1b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:43.192914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702da0fe-1a64-49cd-a945-21c2ceb9ff1b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:43.197662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edecaa-8f4e-4617-88ba-64d05a52b037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:43.198278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edecaa-8f4e-4617-88ba-64d05a52b037 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:43.198556 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edecaa-8f4e-4617-88ba-64d05a52b037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:43.198834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edecaa-8f4e-4617-88ba-64d05a52b037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:43.199517 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edecaa-8f4e-4617-88ba-64d05a52b037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:43.201121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edecaa-8f4e-4617-88ba-64d05a52b037 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:43.211889 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1251/2514] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:43.216394 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a879ac1-2940-425e-8ace-0cd2ff9cdd84 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:43.226650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a879ac1-2940-425e-8ace-0cd2ff9cdd84 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:43.231460 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a09de-cf5c-4dcc-a853-1c32439caced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:43.232061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a09de-cf5c-4dcc-a853-1c32439caced admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:43.232343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a09de-cf5c-4dcc-a853-1c32439caced admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:43.232653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a09de-cf5c-4dcc-a853-1c32439caced admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:43.233298 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a09de-cf5c-4dcc-a853-1c32439caced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:43.234917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a09de-cf5c-4dcc-a853-1c32439caced admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:43.245902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1264/2515] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:43.250939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edecaa-8f4e-4617-88ba-64d05a52b037 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:43.260880 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edecaa-8f4e-4617-88ba-64d05a52b037 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:43.266100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e5697-104b-47d1-b7ff-e2f6ee3bc38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:43.266754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e5697-104b-47d1-b7ff-e2f6ee3bc38b admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:43.267030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e5697-104b-47d1-b7ff-e2f6ee3bc38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:43.267334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e5697-104b-47d1-b7ff-e2f6ee3bc38b admin admin] PATH_INFO: `/v3/projects/11b5b9170b4948b5816fb90564012a8b/users/87f4a0d184574e5ab0f7e3a1039a5d31/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:43.282836 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e5697-104b-47d1-b7ff-e2f6ee3bc38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=11b5b9170b4948b5816fb90564012a8b, user_id=87f4a0d184574e5ab0f7e3a1039a5d31, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:43.285581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e5697-104b-47d1-b7ff-e2f6ee3bc38b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:43.299812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1252/2516] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:43 2022] PUT /identity/v3/projects/11b5b9170b4948b5816fb90564012a8b/users/87f4a0d184574e5ab0f7e3a1039a5d31/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:43.312478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520d1d0-fb2f-42a1-a6d9-0c96c68b5396 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:43.312732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520d1d0-fb2f-42a1-a6d9-0c96c68b5396 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:43.313010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520d1d0-fb2f-42a1-a6d9-0c96c68b5396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:43.344691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d520d1d0-fb2f-42a1-a6d9-0c96c68b5396 None None] MFA Rules not processed for user `87f4a0d184574e5ab0f7e3a1039a5d31`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:43.345944 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d520d1d0-fb2f-42a1-a6d9-0c96c68b5396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:43.414151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1265/2517] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:43.418565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e5697-104b-47d1-b7ff-e2f6ee3bc38b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:43.482241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e5697-104b-47d1-b7ff-e2f6ee3bc38b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:43.493134 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba7662-23dc-444f-8425-592dbec1b390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:43.493776 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba7662-23dc-444f-8425-592dbec1b390 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:43.494056 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba7662-23dc-444f-8425-592dbec1b390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:43.494332 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba7662-23dc-444f-8425-592dbec1b390 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:43.494990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba7662-23dc-444f-8425-592dbec1b390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:43.496957 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba7662-23dc-444f-8425-592dbec1b390 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:43.530106 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1253/2518] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:43 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:43.534373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520d1d0-fb2f-42a1-a6d9-0c96c68b5396 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:43.540581 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:43.548669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520d1d0-fb2f-42a1-a6d9-0c96c68b5396 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:43.630916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523fa19-d00a-43ae-b2c8-52b4a220d01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:43.632072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523fa19-d00a-43ae-b2c8-52b4a220d01c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:43.632205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523fa19-d00a-43ae-b2c8-52b4a220d01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:43.632513 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523fa19-d00a-43ae-b2c8-52b4a220d01c admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:43.633371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523fa19-d00a-43ae-b2c8-52b4a220d01c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:43.635798 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523fa19-d00a-43ae-b2c8-52b4a220d01c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:43.658428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1266/2519] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:43 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:43.662190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba7662-23dc-444f-8425-592dbec1b390 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:43.674121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba7662-23dc-444f-8425-592dbec1b390 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:43.682843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c23408-6533-4717-82be-92fe6fea1819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:43.683581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c23408-6533-4717-82be-92fe6fea1819 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:43.683929 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c23408-6533-4717-82be-92fe6fea1819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:43.684273 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c23408-6533-4717-82be-92fe6fea1819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:43.684996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c23408-6533-4717-82be-92fe6fea1819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:43.687717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c23408-6533-4717-82be-92fe6fea1819 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:43.699357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1254/2520] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:43.704728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523fa19-d00a-43ae-b2c8-52b4a220d01c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:43.716637 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523fa19-d00a-43ae-b2c8-52b4a220d01c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:43.722222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6236f1-d36a-4ac0-8e72-42db710aae8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:43.722991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6236f1-d36a-4ac0-8e72-42db710aae8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:43.723400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6236f1-d36a-4ac0-8e72-42db710aae8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:43.723750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6236f1-d36a-4ac0-8e72-42db710aae8b admin admin] PATH_INFO: `/v3/projects/ea3a330c28e04ad087926d162d43657f/users/12629193f4fd4c46bebadff5a9267c68/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:43.741653 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6236f1-d36a-4ac0-8e72-42db710aae8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea3a330c28e04ad087926d162d43657f, user_id=12629193f4fd4c46bebadff5a9267c68, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:43.744248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6236f1-d36a-4ac0-8e72-42db710aae8b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:43.759728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1267/2521] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:43 2022] PUT /identity/v3/projects/ea3a330c28e04ad087926d162d43657f/users/12629193f4fd4c46bebadff5a9267c68/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:43.764416 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c23408-6533-4717-82be-92fe6fea1819 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:43.776156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c23408-6533-4717-82be-92fe6fea1819 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:43.841153 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a791d8-4fa9-4723-a4db-de35bfffd0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:43.841824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a791d8-4fa9-4723-a4db-de35bfffd0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:43.842781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a791d8-4fa9-4723-a4db-de35bfffd0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:43.843154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a791d8-4fa9-4723-a4db-de35bfffd0e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:43.843864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a791d8-4fa9-4723-a4db-de35bfffd0e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:43.845532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a791d8-4fa9-4723-a4db-de35bfffd0e0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:43.855668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1255/2522] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:21:43.860524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6236f1-d36a-4ac0-8e72-42db710aae8b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:43.871791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6236f1-d36a-4ac0-8e72-42db710aae8b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:43.877067 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d8cd5-fc60-47b2-bc1a-6d3ce085b819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:43.877667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d8cd5-fc60-47b2-bc1a-6d3ce085b819 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:43.877943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d8cd5-fc60-47b2-bc1a-6d3ce085b819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:43.878222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d8cd5-fc60-47b2-bc1a-6d3ce085b819 admin admin] PATH_INFO: `/v3/projects/ea3a330c28e04ad087926d162d43657f/users/12629193f4fd4c46bebadff5a9267c68/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:43.880222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d8cd5-fc60-47b2-bc1a-6d3ce085b819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea3a330c28e04ad087926d162d43657f, user_id=12629193f4fd4c46bebadff5a9267c68, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:43.882983 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d8cd5-fc60-47b2-bc1a-6d3ce085b819 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:43.895267 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1268/2523] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:43 2022] PUT /identity/v3/projects/ea3a330c28e04ad087926d162d43657f/users/12629193f4fd4c46bebadff5a9267c68/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:43.907544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bea9fb-b55f-470d-906c-d68ae0942616 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:43.907832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bea9fb-b55f-470d-906c-d68ae0942616 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:43.908112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bea9fb-b55f-470d-906c-d68ae0942616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:43.915701 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:43.944197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f8bea9fb-b55f-470d-906c-d68ae0942616 None None] MFA Rules not processed for user `12629193f4fd4c46bebadff5a9267c68`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:43.945467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f8bea9fb-b55f-470d-906c-d68ae0942616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:44.121429 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1256/2524] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:44.133831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d8cd5-fc60-47b2-bc1a-6d3ce085b819 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:44.136117 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-402d8cd5-fc60-47b2-bc1a-6d3ce085b819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:44.220995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d8cd5-fc60-47b2-bc1a-6d3ce085b819 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:44.224693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4c91b-8572-46cc-8dbb-6822282a2e5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:44.225337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4c91b-8572-46cc-8dbb-6822282a2e5d service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:44.225616 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4c91b-8572-46cc-8dbb-6822282a2e5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:44.225887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4c91b-8572-46cc-8dbb-6822282a2e5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:44.300079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bea9fb-b55f-470d-906c-d68ae0942616 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:44.301795 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f8bea9fb-b55f-470d-906c-d68ae0942616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:44.315094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4c91b-8572-46cc-8dbb-6822282a2e5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:44.318033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4c91b-8572-46cc-8dbb-6822282a2e5d service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:44.333328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1269/2525] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:44.406022 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bea9fb-b55f-470d-906c-d68ae0942616 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:44.411450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222916d-8f1d-43cf-9fce-cdbcf2a3bd68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:44.412262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222916d-8f1d-43cf-9fce-cdbcf2a3bd68 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:44.412563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222916d-8f1d-43cf-9fce-cdbcf2a3bd68 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:44.412981 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222916d-8f1d-43cf-9fce-cdbcf2a3bd68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:44.423977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e222916d-8f1d-43cf-9fce-cdbcf2a3bd68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:44.437745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222916d-8f1d-43cf-9fce-cdbcf2a3bd68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:44.439788 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222916d-8f1d-43cf-9fce-cdbcf2a3bd68 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:44.454688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1257/2526] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:21:46.400705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4c91b-8572-46cc-8dbb-6822282a2e5d service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:46.402408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9af4c91b-8572-46cc-8dbb-6822282a2e5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:46.416052 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:46.475950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4c91b-8572-46cc-8dbb-6822282a2e5d service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:46.481898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a63f47-2b38-4c97-8561-3ce0d37d7071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:46.482664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a63f47-2b38-4c97-8561-3ce0d37d7071 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:46.483149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a63f47-2b38-4c97-8561-3ce0d37d7071 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:46.483689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a63f47-2b38-4c97-8561-3ce0d37d7071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:46.485542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-25a63f47-2b38-4c97-8561-3ce0d37d7071 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:46.556562 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a63f47-2b38-4c97-8561-3ce0d37d7071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:46.558291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a63f47-2b38-4c97-8561-3ce0d37d7071 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:46.568902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1270/2527] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:46.964548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222916d-8f1d-43cf-9fce-cdbcf2a3bd68 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:46.976417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222916d-8f1d-43cf-9fce-cdbcf2a3bd68 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:46.981225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:46.981994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:46.982372 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:46.983328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] PATH_INFO: `/v3/users/87f4a0d184574e5ab0f7e3a1039a5d31` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:46.985159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f4a0d184574e5ab0f7e3a1039a5d31)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:46.987325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:47.052638 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f4a0d184574e5ab0f7e3a1039a5d31'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:47.063012 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f4a0d184574e5ab0f7e3a1039a5d31'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:47.078979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f4a0d184574e5ab0f7e3a1039a5d31'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:47.079895 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f4a0d184574e5ab0f7e3a1039a5d31'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:47.081350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:47.096292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f4a0d184574e5ab0f7e3a1039a5d31'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:47.105267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1258/2528] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:46 2022] DELETE /identity/v3/users/87f4a0d184574e5ab0f7e3a1039a5d31 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:47.114714 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20ce18-2a34-4415-8718-cef6a6f531d6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:47.115003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20ce18-2a34-4415-8718-cef6a6f531d6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:47.115307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20ce18-2a34-4415-8718-cef6a6f531d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:47.147425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7a20ce18-2a34-4415-8718-cef6a6f531d6 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:47.148700 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7a20ce18-2a34-4415-8718-cef6a6f531d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:47.229167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1271/2529] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:21:47.240016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:47.241584 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:47.306118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78b220-ca7c-4205-91e9-aad58d22f5c9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:47.311202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac385c4-1bb7-40e4-a903-b401b1aa2c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:47.312267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac385c4-1bb7-40e4-a903-b401b1aa2c94 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:47.312637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac385c4-1bb7-40e4-a903-b401b1aa2c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:47.313004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac385c4-1bb7-40e4-a903-b401b1aa2c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:47.325139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac385c4-1bb7-40e4-a903-b401b1aa2c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:47.327393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac385c4-1bb7-40e4-a903-b401b1aa2c94 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:47.338809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1259/2530] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:47.657721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20ce18-2a34-4415-8718-cef6a6f531d6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:47.659196 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7a20ce18-2a34-4415-8718-cef6a6f531d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:47.692903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20ce18-2a34-4415-8718-cef6a6f531d6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:47.700784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bc8b5-d70e-4b3e-ae58-4c589e404751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:47.701580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bc8b5-d70e-4b3e-ae58-4c589e404751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:47.701986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bc8b5-d70e-4b3e-ae58-4c589e404751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:47.702170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bc8b5-d70e-4b3e-ae58-4c589e404751 admin admin] PATH_INFO: `/v3/projects/11b5b9170b4948b5816fb90564012a8b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:47.703550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439bc8b5-d70e-4b3e-ae58-4c589e404751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11b5b9170b4948b5816fb90564012a8b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:47.706024 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439bc8b5-d70e-4b3e-ae58-4c589e404751 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:47.774346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-439bc8b5-d70e-4b3e-ae58-4c589e404751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11b5b9170b4948b5816fb90564012a8b', 'request_id': 'req-439bc8b5-d70e-4b3e-ae58-4c589e404751'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:47.778498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:47.798050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-439bc8b5-d70e-4b3e-ae58-4c589e404751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11b5b9170b4948b5816fb90564012a8b', 'request_id': 'req-439bc8b5-d70e-4b3e-ae58-4c589e404751'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:47.806279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-439bc8b5-d70e-4b3e-ae58-4c589e404751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11b5b9170b4948b5816fb90564012a8b', 'request_id': 'req-439bc8b5-d70e-4b3e-ae58-4c589e404751'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:47.819796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-439bc8b5-d70e-4b3e-ae58-4c589e404751 admin admin] The token cache is being invalidate because project 11b5b9170b4948b5816fb90564012a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:47.820396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-439bc8b5-d70e-4b3e-ae58-4c589e404751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:47.821998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-439bc8b5-d70e-4b3e-ae58-4c589e404751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:47.824323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1272/2531] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:47 2022] DELETE /identity/v3/projects/11b5b9170b4948b5816fb90564012a8b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:47.829019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac385c4-1bb7-40e4-a903-b401b1aa2c94 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:47.841762 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac385c4-1bb7-40e4-a903-b401b1aa2c94 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:47.971255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e4d39-59c8-45a5-a907-125bfe686d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:47.971924 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e4d39-59c8-45a5-a907-125bfe686d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:47.972211 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e4d39-59c8-45a5-a907-125bfe686d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:47.972498 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e4d39-59c8-45a5-a907-125bfe686d47 admin admin] PATH_INFO: `/v3/users/12629193f4fd4c46bebadff5a9267c68` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:47.974953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e4d39-59c8-45a5-a907-125bfe686d47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12629193f4fd4c46bebadff5a9267c68)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:47.980944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e4d39-59c8-45a5-a907-125bfe686d47 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:48.084925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0c3e4d39-59c8-45a5-a907-125bfe686d47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12629193f4fd4c46bebadff5a9267c68'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.112159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0c3e4d39-59c8-45a5-a907-125bfe686d47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12629193f4fd4c46bebadff5a9267c68'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.133137 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0c3e4d39-59c8-45a5-a907-125bfe686d47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12629193f4fd4c46bebadff5a9267c68'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.133919 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0c3e4d39-59c8-45a5-a907-125bfe686d47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12629193f4fd4c46bebadff5a9267c68'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.142258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0c3e4d39-59c8-45a5-a907-125bfe686d47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12629193f4fd4c46bebadff5a9267c68'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.151821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1260/2532] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:47 2022] DELETE /identity/v3/users/12629193f4fd4c46bebadff5a9267c68 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:48.174733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b839ab9-1ca9-4cd8-b4d0-fc30f0e09a56 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:48.175251 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b839ab9-1ca9-4cd8-b4d0-fc30f0e09a56 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:48.175706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b839ab9-1ca9-4cd8-b4d0-fc30f0e09a56 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:48.177654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1273/2533] 10.222.0.17 () {54 vars in 961 bytes} [Fri Sep 9 13:21:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:21:48.183433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bc3eb-64c6-44ff-b192-a419b3672bdf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:48.183726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bc3eb-64c6-44ff-b192-a419b3672bdf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:48.184006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bc3eb-64c6-44ff-b192-a419b3672bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:48.220022 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ad7bc3eb-64c6-44ff-b192-a419b3672bdf None None] MFA Rules not processed for user `dbc7cef7b714463697dc1ee507e60be1`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:48.221606 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ad7bc3eb-64c6-44ff-b192-a419b3672bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:48.229799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:48.260347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b839ab9-1ca9-4cd8-b4d0-fc30f0e09a56 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:48.262903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0b839ab9-1ca9-4cd8-b4d0-fc30f0e09a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:48.352314 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1261/2534] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Sep 9 13:21:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:48.367618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bc3eb-64c6-44ff-b192-a419b3672bdf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:48.378512 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b839ab9-1ca9-4cd8-b4d0-fc30f0e09a56 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:48.382189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bc3eb-64c6-44ff-b192-a419b3672bdf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:48.384257 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:48.385131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:48.385616 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:48.386093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] PATH_INFO: `/v3/users/b754de037a8148358efb03ffa39786d4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:48.387482 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=b754de037a8148358efb03ffa39786d4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:48.387617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c9cc5-4891-4ed8-a028-19f28558ec4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:48.388103 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c9cc5-4891-4ed8-a028-19f28558ec4a service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:48.388378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c9cc5-4891-4ed8-a028-19f28558ec4a service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:48.388650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c9cc5-4891-4ed8-a028-19f28558ec4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:48.390827 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:48.391395 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-045c9cc5-4891-4ed8-a028-19f28558ec4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:48.406764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c9cc5-4891-4ed8-a028-19f28558ec4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:48.408423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c9cc5-4891-4ed8-a028-19f28558ec4a service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:48.427171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1262/2535] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:21:48.459506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b754de037a8148358efb03ffa39786d4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.469606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b754de037a8148358efb03ffa39786d4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.470448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b754de037a8148358efb03ffa39786d4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.490277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b754de037a8148358efb03ffa39786d4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.500552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b754de037a8148358efb03ffa39786d4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.508188 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1274/2536] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:48 2022] DELETE /identity/v3/users/b754de037a8148358efb03ffa39786d4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:48.530714 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c9cc5-4891-4ed8-a028-19f28558ec4a service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:48.532279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-045c9cc5-4891-4ed8-a028-19f28558ec4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:48.556728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c9cc5-4891-4ed8-a028-19f28558ec4a service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:48.632869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:48.633552 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:48.633832 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:48.634101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] PATH_INFO: `/v3/projects/ea3a330c28e04ad087926d162d43657f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:48.635269 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea3a330c28e04ad087926d162d43657f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:48.637223 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:48.714332 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea3a330c28e04ad087926d162d43657f', 'request_id': 'req-18fd3a57-c674-4cfb-8821-67371de39c91'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.722906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea3a330c28e04ad087926d162d43657f', 'request_id': 'req-18fd3a57-c674-4cfb-8821-67371de39c91'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.736006 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea3a330c28e04ad087926d162d43657f', 'request_id': 'req-18fd3a57-c674-4cfb-8821-67371de39c91'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.761845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] The token cache is being invalidate because project ea3a330c28e04ad087926d162d43657f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:48.762459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.763288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:48.767347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1263/2537] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:48 2022] DELETE /identity/v3/projects/ea3a330c28e04ad087926d162d43657f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:48.776677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:48.778378 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:48.797408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef3fd9-9937-473d-b054-f1d8822d5ead admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:48.956371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:48.957356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:48.957830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:48.958316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] PATH_INFO: `/v3/projects/e7c4b6192be646b38d7757960e02feda` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:48.960097 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c4b6192be646b38d7757960e02feda)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:48.962955 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:48.965974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:49.040163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c4b6192be646b38d7757960e02feda', 'request_id': 'req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.052480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c4b6192be646b38d7757960e02feda', 'request_id': 'req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.060072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c4b6192be646b38d7757960e02feda', 'request_id': 'req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.073163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] The token cache is being invalidate because project e7c4b6192be646b38d7757960e02feda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:49.073539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.074289 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.076371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1275/2538] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:48 2022] DELETE /identity/v3/projects/e7c4b6192be646b38d7757960e02feda => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:49.080342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:49.081843 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:49.096131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd3a57-c674-4cfb-8821-67371de39c91 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:49.117433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:49.169021 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:49.169666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:49.169940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:49.170944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] PATH_INFO: `/v3/users/d8eee1ec9ecd4dfbae1864be468894f1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:49.172760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8eee1ec9ecd4dfbae1864be468894f1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:49.175004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:49.231420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8eee1ec9ecd4dfbae1864be468894f1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.241169 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8eee1ec9ecd4dfbae1864be468894f1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.259950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8eee1ec9ecd4dfbae1864be468894f1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.260727 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8eee1ec9ecd4dfbae1864be468894f1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.270588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8eee1ec9ecd4dfbae1864be468894f1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.279079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1264/2539] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:49 2022] DELETE /identity/v3/users/d8eee1ec9ecd4dfbae1864be468894f1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:49.465578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:49.467444 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:49.481276 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61469a4e-16cb-4fb9-bc43-0e6f106e7e88 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:49.544424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:49.544982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:49.545914 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:49.546198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] PATH_INFO: `/v3/projects/164df8b44ce844e58f7d168409990608` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:49.547424 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=164df8b44ce844e58f7d168409990608)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:49.549510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:49.615936 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '164df8b44ce844e58f7d168409990608', 'request_id': 'req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.626906 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '164df8b44ce844e58f7d168409990608', 'request_id': 'req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.635153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '164df8b44ce844e58f7d168409990608', 'request_id': 'req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.649018 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] The token cache is being invalidate because project 164df8b44ce844e58f7d168409990608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:49.649470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.650217 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:49.655097 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1276/2540] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:49 2022] DELETE /identity/v3/projects/164df8b44ce844e58f7d168409990608 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:49.928430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:49.929869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:50.011234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8f41f-7ab5-4c22-8fc5-e38d0d99297d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:50.066735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:50.102088 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6166b-fba5-4be0-9584-66ec7150d7de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:50.103699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6166b-fba5-4be0-9584-66ec7150d7de service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:50.104074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6166b-fba5-4be0-9584-66ec7150d7de service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:50.104353 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6166b-fba5-4be0-9584-66ec7150d7de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:50.106457 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e3a6166b-fba5-4be0-9584-66ec7150d7de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:50.186565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6166b-fba5-4be0-9584-66ec7150d7de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:50.188276 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6166b-fba5-4be0-9584-66ec7150d7de service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:50.203284 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1265/2541] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:50.808425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:50.810545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:50.840883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:50.884343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c7a0a-f618-4239-87a6-d4aeb34e58e9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:50.889435 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aac164-1635-49e8-a2f8-b618b58b307e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:50.890259 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aac164-1635-49e8-a2f8-b618b58b307e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:50.890542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aac164-1635-49e8-a2f8-b618b58b307e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:50.890949 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aac164-1635-49e8-a2f8-b618b58b307e admin admin] PATH_INFO: `/v3/users/c31bb28e38864eff99fdd6938b1b89a0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:50.892122 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aac164-1635-49e8-a2f8-b618b58b307e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c31bb28e38864eff99fdd6938b1b89a0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:50.894185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aac164-1635-49e8-a2f8-b618b58b307e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:50.943606 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-21aac164-1635-49e8-a2f8-b618b58b307e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c31bb28e38864eff99fdd6938b1b89a0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:50.952823 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-21aac164-1635-49e8-a2f8-b618b58b307e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c31bb28e38864eff99fdd6938b1b89a0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:50.953536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-21aac164-1635-49e8-a2f8-b618b58b307e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c31bb28e38864eff99fdd6938b1b89a0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:50.967531 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-21aac164-1635-49e8-a2f8-b618b58b307e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c31bb28e38864eff99fdd6938b1b89a0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:50.976505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-21aac164-1635-49e8-a2f8-b618b58b307e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c31bb28e38864eff99fdd6938b1b89a0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:50.983242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1277/2542] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:50 2022] DELETE /identity/v3/users/c31bb28e38864eff99fdd6938b1b89a0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:51.200642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6166b-fba5-4be0-9584-66ec7150d7de service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:51.214764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6166b-fba5-4be0-9584-66ec7150d7de service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:51.276759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc457d90-39dc-4b96-9517-232f8e1ffbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:51.277500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc457d90-39dc-4b96-9517-232f8e1ffbb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:51.277778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc457d90-39dc-4b96-9517-232f8e1ffbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:51.278053 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc457d90-39dc-4b96-9517-232f8e1ffbb3 admin admin] PATH_INFO: `/v3/projects/6f6fd89d622647f49bd35dcaf8ceca55` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:51.279251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc457d90-39dc-4b96-9517-232f8e1ffbb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f6fd89d622647f49bd35dcaf8ceca55)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:51.281896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc457d90-39dc-4b96-9517-232f8e1ffbb3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:51.330397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dc457d90-39dc-4b96-9517-232f8e1ffbb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f6fd89d622647f49bd35dcaf8ceca55', 'request_id': 'req-dc457d90-39dc-4b96-9517-232f8e1ffbb3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:51.336412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dc457d90-39dc-4b96-9517-232f8e1ffbb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f6fd89d622647f49bd35dcaf8ceca55', 'request_id': 'req-dc457d90-39dc-4b96-9517-232f8e1ffbb3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:51.343394 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dc457d90-39dc-4b96-9517-232f8e1ffbb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f6fd89d622647f49bd35dcaf8ceca55', 'request_id': 'req-dc457d90-39dc-4b96-9517-232f8e1ffbb3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:51.355895 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dc457d90-39dc-4b96-9517-232f8e1ffbb3 admin admin] The token cache is being invalidate because project 6f6fd89d622647f49bd35dcaf8ceca55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:51.356191 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dc457d90-39dc-4b96-9517-232f8e1ffbb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:51.356816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-dc457d90-39dc-4b96-9517-232f8e1ffbb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:51.358922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1266/2543] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:51 2022] DELETE /identity/v3/projects/6f6fd89d622647f49bd35dcaf8ceca55 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:51.363985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ea4b1-9bdb-49c5-9ae3-31fb5ef3fd80 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:51.364275 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ea4b1-9bdb-49c5-9ae3-31fb5ef3fd80 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:51.364552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ea4b1-9bdb-49c5-9ae3-31fb5ef3fd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:51.384102 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5b8ea4b1-9bdb-49c5-9ae3-31fb5ef3fd80 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:51.385315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5b8ea4b1-9bdb-49c5-9ae3-31fb5ef3fd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:51.444234 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1278/2544] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:51.452381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b81be2-2705-48f1-82fb-afb9aa94fc50 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:51.452668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b81be2-2705-48f1-82fb-afb9aa94fc50 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:51.452944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b81be2-2705-48f1-82fb-afb9aa94fc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:51.473260 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-54b81be2-2705-48f1-82fb-afb9aa94fc50 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:51.474452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-54b81be2-2705-48f1-82fb-afb9aa94fc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:51.483347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1267/2545] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:51.487549 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ea4b1-9bdb-49c5-9ae3-31fb5ef3fd80 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:51.497228 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ea4b1-9bdb-49c5-9ae3-31fb5ef3fd80 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:51.502255 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b9f4c-6906-49f7-a223-2914c039b58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:51.502855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b9f4c-6906-49f7-a223-2914c039b58d admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:51.503176 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b9f4c-6906-49f7-a223-2914c039b58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:51.503477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b9f4c-6906-49f7-a223-2914c039b58d admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:51.504126 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b9f4c-6906-49f7-a223-2914c039b58d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:21:51.504433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b9f4c-6906-49f7-a223-2914c039b58d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:51.506021 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b9f4c-6906-49f7-a223-2914c039b58d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:51.517667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1279/2546] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:21:51.521777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b81be2-2705-48f1-82fb-afb9aa94fc50 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:51.526145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:51.533595 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b81be2-2705-48f1-82fb-afb9aa94fc50 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:51.539253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4431f7-3639-4637-b341-de58dc2d12f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:51.539858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4431f7-3639-4637-b341-de58dc2d12f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:51.540140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4431f7-3639-4637-b341-de58dc2d12f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:51.540410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4431f7-3639-4637-b341-de58dc2d12f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:51.541046 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4431f7-3639-4637-b341-de58dc2d12f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:51.542953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4431f7-3639-4637-b341-de58dc2d12f7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:51.571632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1268/2547] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:51 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:51.575955 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b9f4c-6906-49f7-a223-2914c039b58d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:51.586690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b9f4c-6906-49f7-a223-2914c039b58d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:51.613481 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:51.647201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3cd18-59ee-4e50-bd59-ca8e09379493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:51.647806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3cd18-59ee-4e50-bd59-ca8e09379493 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:51.648090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3cd18-59ee-4e50-bd59-ca8e09379493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:51.648361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3cd18-59ee-4e50-bd59-ca8e09379493 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:51.648991 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3cd18-59ee-4e50-bd59-ca8e09379493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:51.651067 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3cd18-59ee-4e50-bd59-ca8e09379493 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:51.673859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1280/2548] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:51 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:21:51.677983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4431f7-3639-4637-b341-de58dc2d12f7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:51.688501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4431f7-3639-4637-b341-de58dc2d12f7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:51.693870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98949207-42b2-4a82-86dc-0bbd387fd589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:51.694120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98949207-42b2-4a82-86dc-0bbd387fd589 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:51.694399 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98949207-42b2-4a82-86dc-0bbd387fd589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:51.694673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98949207-42b2-4a82-86dc-0bbd387fd589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:51.695350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98949207-42b2-4a82-86dc-0bbd387fd589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:51.697000 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98949207-42b2-4a82-86dc-0bbd387fd589 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:51.708765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1269/2549] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:51.712964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3cd18-59ee-4e50-bd59-ca8e09379493 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:51.723994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3cd18-59ee-4e50-bd59-ca8e09379493 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:51.728785 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7e405-be2c-457d-b6fa-9659bbe00bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:51.729553 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7e405-be2c-457d-b6fa-9659bbe00bff admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:51.729883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7e405-be2c-457d-b6fa-9659bbe00bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:51.735149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7e405-be2c-457d-b6fa-9659bbe00bff admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:51.735908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7e405-be2c-457d-b6fa-9659bbe00bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:51.737497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7e405-be2c-457d-b6fa-9659bbe00bff admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:51.749287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1281/2550] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:21:51.753491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98949207-42b2-4a82-86dc-0bbd387fd589 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:51.763328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98949207-42b2-4a82-86dc-0bbd387fd589 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:51.768619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47310cb-e0e1-4730-af99-93e9e0425479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:51.769265 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47310cb-e0e1-4730-af99-93e9e0425479 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:51.769543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47310cb-e0e1-4730-af99-93e9e0425479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:51.769818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47310cb-e0e1-4730-af99-93e9e0425479 admin admin] PATH_INFO: `/v3/projects/92288a0a820448ebb59f681cc69436bf/users/caac4a78a3ee48b9bb554606304bf468/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:51.784613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47310cb-e0e1-4730-af99-93e9e0425479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92288a0a820448ebb59f681cc69436bf, user_id=caac4a78a3ee48b9bb554606304bf468, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:51.787309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47310cb-e0e1-4730-af99-93e9e0425479 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:51.800867 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1270/2551] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:51 2022] PUT /identity/v3/projects/92288a0a820448ebb59f681cc69436bf/users/caac4a78a3ee48b9bb554606304bf468/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:51.814015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446081f7-8c99-4975-b9dc-c2833afd98c1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:51.814492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446081f7-8c99-4975-b9dc-c2833afd98c1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:51.814663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446081f7-8c99-4975-b9dc-c2833afd98c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:51.848427 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-446081f7-8c99-4975-b9dc-c2833afd98c1 None None] MFA Rules not processed for user `caac4a78a3ee48b9bb554606304bf468`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:51.849899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-446081f7-8c99-4975-b9dc-c2833afd98c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:51.923515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1282/2552] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:51.935008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47310cb-e0e1-4730-af99-93e9e0425479 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:51.937441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d47310cb-e0e1-4730-af99-93e9e0425479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:52.012010 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47310cb-e0e1-4730-af99-93e9e0425479 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:52.019148 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed7bb4-4adc-41b1-b298-4b7b7a5f10c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:52.019879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed7bb4-4adc-41b1-b298-4b7b7a5f10c1 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:52.020274 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed7bb4-4adc-41b1-b298-4b7b7a5f10c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:52.020628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed7bb4-4adc-41b1-b298-4b7b7a5f10c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:52.031589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed7bb4-4adc-41b1-b298-4b7b7a5f10c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:52.033799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed7bb4-4adc-41b1-b298-4b7b7a5f10c1 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:52.045077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1271/2553] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:54.722995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446081f7-8c99-4975-b9dc-c2833afd98c1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:54.784235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446081f7-8c99-4975-b9dc-c2833afd98c1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:54.788845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:54.789470 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:54.789748 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:54.790019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] PATH_INFO: `/v3/users/caac4a78a3ee48b9bb554606304bf468` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:54.791155 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caac4a78a3ee48b9bb554606304bf468)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:54.793069 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:54.833121 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:54.839426 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caac4a78a3ee48b9bb554606304bf468'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:54.847787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caac4a78a3ee48b9bb554606304bf468'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:54.848491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caac4a78a3ee48b9bb554606304bf468'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:54.860680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caac4a78a3ee48b9bb554606304bf468'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:54.867631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caac4a78a3ee48b9bb554606304bf468'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:54.875614 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1283/2554] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:54 2022] DELETE /identity/v3/users/caac4a78a3ee48b9bb554606304bf468 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:54.880350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede61f20-6994-457c-93b0-23115a35ea9d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:54.880350 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede61f20-6994-457c-93b0-23115a35ea9d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:54.880853 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede61f20-6994-457c-93b0-23115a35ea9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:54.900108 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ede61f20-6994-457c-93b0-23115a35ea9d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:54.901325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ede61f20-6994-457c-93b0-23115a35ea9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:54.963037 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1272/2555] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:54.973293 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:54.974738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:55.047224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510b552-c6f6-495a-8825-4eec71990024 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:55.050970 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911c0ac-4a8c-4bbc-bcb8-65ae9219ffa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:55.051612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911c0ac-4a8c-4bbc-bcb8-65ae9219ffa0 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:55.051892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911c0ac-4a8c-4bbc-bcb8-65ae9219ffa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:55.052166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911c0ac-4a8c-4bbc-bcb8-65ae9219ffa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:55.061360 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911c0ac-4a8c-4bbc-bcb8-65ae9219ffa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:55.062797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911c0ac-4a8c-4bbc-bcb8-65ae9219ffa0 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:55.072278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1284/2556] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:55.348180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede61f20-6994-457c-93b0-23115a35ea9d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:55.349865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ede61f20-6994-457c-93b0-23115a35ea9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:55.362884 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede61f20-6994-457c-93b0-23115a35ea9d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:55.367764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981288fd-4034-479f-a708-ff293e088b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:55.368406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981288fd-4034-479f-a708-ff293e088b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:55.368688 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981288fd-4034-479f-a708-ff293e088b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:55.368993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981288fd-4034-479f-a708-ff293e088b58 admin admin] PATH_INFO: `/v3/projects/92288a0a820448ebb59f681cc69436bf` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:55.370210 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981288fd-4034-479f-a708-ff293e088b58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92288a0a820448ebb59f681cc69436bf)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:55.372207 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981288fd-4034-479f-a708-ff293e088b58 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:55.403842 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:55.423281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-981288fd-4034-479f-a708-ff293e088b58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92288a0a820448ebb59f681cc69436bf', 'request_id': 'req-981288fd-4034-479f-a708-ff293e088b58'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:55.428746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-981288fd-4034-479f-a708-ff293e088b58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92288a0a820448ebb59f681cc69436bf', 'request_id': 'req-981288fd-4034-479f-a708-ff293e088b58'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:55.435175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-981288fd-4034-479f-a708-ff293e088b58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92288a0a820448ebb59f681cc69436bf', 'request_id': 'req-981288fd-4034-479f-a708-ff293e088b58'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:55.447361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-981288fd-4034-479f-a708-ff293e088b58 admin admin] The token cache is being invalidate because project 92288a0a820448ebb59f681cc69436bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:55.447649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-981288fd-4034-479f-a708-ff293e088b58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:55.448270 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-981288fd-4034-479f-a708-ff293e088b58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:55.449835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1273/2557] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:55 2022] DELETE /identity/v3/projects/92288a0a820448ebb59f681cc69436bf => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:55.455639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a730d-c1a5-462b-8c3c-60d3e6ae4d9c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:55.455927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a730d-c1a5-462b-8c3c-60d3e6ae4d9c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:55.456201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a730d-c1a5-462b-8c3c-60d3e6ae4d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:55.476027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4f8a730d-c1a5-462b-8c3c-60d3e6ae4d9c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:55.477229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4f8a730d-c1a5-462b-8c3c-60d3e6ae4d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:55.536897 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1285/2558] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:55.544835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b163d-b8e1-441d-9d7f-094769942acf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:55.545529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b163d-b8e1-441d-9d7f-094769942acf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:55.545812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b163d-b8e1-441d-9d7f-094769942acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:55.565709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-969b163d-b8e1-441d-9d7f-094769942acf None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:55.566918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-969b163d-b8e1-441d-9d7f-094769942acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:55.574814 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1274/2559] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:55.579747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a730d-c1a5-462b-8c3c-60d3e6ae4d9c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:55.589563 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a730d-c1a5-462b-8c3c-60d3e6ae4d9c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:55.595292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e2136-f669-4f68-a12c-2019452e31c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:55.595906 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e2136-f669-4f68-a12c-2019452e31c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:55.596206 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e2136-f669-4f68-a12c-2019452e31c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:55.596479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e2136-f669-4f68-a12c-2019452e31c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:55.597129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e2136-f669-4f68-a12c-2019452e31c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:21:55.597435 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e2136-f669-4f68-a12c-2019452e31c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:55.599033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e2136-f669-4f68-a12c-2019452e31c5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:55.611243 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1286/2560] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:21:55.615764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b163d-b8e1-441d-9d7f-094769942acf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:55.625532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b163d-b8e1-441d-9d7f-094769942acf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:55.630753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b14a74-3804-4039-83a3-2b795ba8667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:55.631408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b14a74-3804-4039-83a3-2b795ba8667c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:55.631689 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b14a74-3804-4039-83a3-2b795ba8667c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:55.631963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b14a74-3804-4039-83a3-2b795ba8667c admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:55.632589 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b14a74-3804-4039-83a3-2b795ba8667c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:55.634581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b14a74-3804-4039-83a3-2b795ba8667c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:55.662622 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1275/2561] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:55 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:55.666791 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e2136-f669-4f68-a12c-2019452e31c5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:55.677180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e2136-f669-4f68-a12c-2019452e31c5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:55.721543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:55.735888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fe2c5-f2fb-47e9-8afc-cf66ec49447e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:55.736498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fe2c5-f2fb-47e9-8afc-cf66ec49447e admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:55.736762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fe2c5-f2fb-47e9-8afc-cf66ec49447e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:55.737032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fe2c5-f2fb-47e9-8afc-cf66ec49447e admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:55.737657 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fe2c5-f2fb-47e9-8afc-cf66ec49447e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:55.739611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fe2c5-f2fb-47e9-8afc-cf66ec49447e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:55.761277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1287/2562] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:55 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:55.766294 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b14a74-3804-4039-83a3-2b795ba8667c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:55.777258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b14a74-3804-4039-83a3-2b795ba8667c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:55.781895 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d94c5-c8fc-4e53-9d90-e7a65c395ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:55.782482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d94c5-c8fc-4e53-9d90-e7a65c395ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:55.782761 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d94c5-c8fc-4e53-9d90-e7a65c395ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:55.783030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d94c5-c8fc-4e53-9d90-e7a65c395ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:55.783702 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d94c5-c8fc-4e53-9d90-e7a65c395ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:55.785290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d94c5-c8fc-4e53-9d90-e7a65c395ff4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:55.795075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1276/2563] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:55.799165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fe2c5-f2fb-47e9-8afc-cf66ec49447e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:55.809351 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fe2c5-f2fb-47e9-8afc-cf66ec49447e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:55.815426 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac440d-5c2c-40c4-b781-c6a6d623ce76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:55.815955 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac440d-5c2c-40c4-b781-c6a6d623ce76 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:55.816233 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac440d-5c2c-40c4-b781-c6a6d623ce76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:55.816528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac440d-5c2c-40c4-b781-c6a6d623ce76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:55.817166 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac440d-5c2c-40c4-b781-c6a6d623ce76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:55.818785 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac440d-5c2c-40c4-b781-c6a6d623ce76 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:55.830356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1288/2564] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:55.834472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d94c5-c8fc-4e53-9d90-e7a65c395ff4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:55.845761 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d94c5-c8fc-4e53-9d90-e7a65c395ff4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:55.850938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7841193-7bcd-4ba0-8b10-8cb366cc61df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:55.851628 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7841193-7bcd-4ba0-8b10-8cb366cc61df admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:55.851937 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7841193-7bcd-4ba0-8b10-8cb366cc61df admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:55.852218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7841193-7bcd-4ba0-8b10-8cb366cc61df admin admin] PATH_INFO: `/v3/projects/e7c2f682d0b54c62b5b3c205eddc94d4/users/a6785496679a44d49afee7dc806a7ddf/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:55.866737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7841193-7bcd-4ba0-8b10-8cb366cc61df admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c2f682d0b54c62b5b3c205eddc94d4, user_id=a6785496679a44d49afee7dc806a7ddf, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:55.869433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7841193-7bcd-4ba0-8b10-8cb366cc61df admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:55.883571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1277/2565] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:55 2022] PUT /identity/v3/projects/e7c2f682d0b54c62b5b3c205eddc94d4/users/a6785496679a44d49afee7dc806a7ddf/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:55.895434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85743e09-42b7-422b-8634-89fc898544c8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:55.895722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85743e09-42b7-422b-8634-89fc898544c8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:55.895998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85743e09-42b7-422b-8634-89fc898544c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:55.923924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-85743e09-42b7-422b-8634-89fc898544c8 None None] MFA Rules not processed for user `a6785496679a44d49afee7dc806a7ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:55.925241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-85743e09-42b7-422b-8634-89fc898544c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:55.990766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1289/2566] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:21:56.002560 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7841193-7bcd-4ba0-8b10-8cb366cc61df admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:56.004254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e7841193-7bcd-4ba0-8b10-8cb366cc61df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:56.081092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7841193-7bcd-4ba0-8b10-8cb366cc61df admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:56.087728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac5989-555d-46c1-b9d0-73231156b5ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:56.088386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac5989-555d-46c1-b9d0-73231156b5ed service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:56.088664 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac5989-555d-46c1-b9d0-73231156b5ed service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:56.089064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac5989-555d-46c1-b9d0-73231156b5ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:56.098408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac5989-555d-46c1-b9d0-73231156b5ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:56.100064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac5989-555d-46c1-b9d0-73231156b5ed service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:56.112991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1278/2567] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:56.321303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85743e09-42b7-422b-8634-89fc898544c8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:56.383975 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85743e09-42b7-422b-8634-89fc898544c8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:56.388471 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:56.389168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:56.389446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:56.389718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] PATH_INFO: `/v3/users/a6785496679a44d49afee7dc806a7ddf` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:56.390848 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6785496679a44d49afee7dc806a7ddf)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:56.392794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:56.440844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6785496679a44d49afee7dc806a7ddf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:56.448930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6785496679a44d49afee7dc806a7ddf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:56.449544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6785496679a44d49afee7dc806a7ddf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:56.451582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:56.464373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6785496679a44d49afee7dc806a7ddf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:56.473066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6785496679a44d49afee7dc806a7ddf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:56.479296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1290/2568] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:21:56 2022] DELETE /identity/v3/users/a6785496679a44d49afee7dc806a7ddf => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:56.484349 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01042ee-763d-4e7d-8522-4a800cf70ae9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:56.484631 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01042ee-763d-4e7d-8522-4a800cf70ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:56.484899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01042ee-763d-4e7d-8522-4a800cf70ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:56.504485 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c01042ee-763d-4e7d-8522-4a800cf70ae9 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:56.505617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c01042ee-763d-4e7d-8522-4a800cf70ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:56.536440 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:56.567845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1279/2569] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:56.577634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:56.579142 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:56.640782 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf0e5c-852d-4709-8634-286766776455 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:56.644445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22bd26-8c83-49df-bc5d-b69af4f5126d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:56.645047 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22bd26-8c83-49df-bc5d-b69af4f5126d service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:56.645326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22bd26-8c83-49df-bc5d-b69af4f5126d service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:56.645597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22bd26-8c83-49df-bc5d-b69af4f5126d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:56.655003 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22bd26-8c83-49df-bc5d-b69af4f5126d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:56.656479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22bd26-8c83-49df-bc5d-b69af4f5126d service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:56.666222 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1291/2570] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:21:56.944462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01042ee-763d-4e7d-8522-4a800cf70ae9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:56.945941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c01042ee-763d-4e7d-8522-4a800cf70ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:56.959198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01042ee-763d-4e7d-8522-4a800cf70ae9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:56.963870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899d30a-758b-4cd7-8c98-5daec970ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:56.964581 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899d30a-758b-4cd7-8c98-5daec970ec45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:56.965402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899d30a-758b-4cd7-8c98-5daec970ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:56.965929 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899d30a-758b-4cd7-8c98-5daec970ec45 admin admin] PATH_INFO: `/v3/projects/e7c2f682d0b54c62b5b3c205eddc94d4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:56.967993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899d30a-758b-4cd7-8c98-5daec970ec45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c2f682d0b54c62b5b3c205eddc94d4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:56.970078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899d30a-758b-4cd7-8c98-5daec970ec45 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:57.019247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9899d30a-758b-4cd7-8c98-5daec970ec45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c2f682d0b54c62b5b3c205eddc94d4', 'request_id': 'req-9899d30a-758b-4cd7-8c98-5daec970ec45'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:57.024750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9899d30a-758b-4cd7-8c98-5daec970ec45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c2f682d0b54c62b5b3c205eddc94d4', 'request_id': 'req-9899d30a-758b-4cd7-8c98-5daec970ec45'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:57.031808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9899d30a-758b-4cd7-8c98-5daec970ec45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c2f682d0b54c62b5b3c205eddc94d4', 'request_id': 'req-9899d30a-758b-4cd7-8c98-5daec970ec45'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:57.044671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9899d30a-758b-4cd7-8c98-5daec970ec45 admin admin] The token cache is being invalidate because project e7c2f682d0b54c62b5b3c205eddc94d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:21:57.044986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9899d30a-758b-4cd7-8c98-5daec970ec45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:57.045685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9899d30a-758b-4cd7-8c98-5daec970ec45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:21:57.047238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1280/2571] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:21:56 2022] DELETE /identity/v3/projects/e7c2f682d0b54c62b5b3c205eddc94d4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:57.052154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10cded-60f7-47f8-800b-fb1b9b078776 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:57.052643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10cded-60f7-47f8-800b-fb1b9b078776 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:57.052745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10cded-60f7-47f8-800b-fb1b9b078776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:57.079548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-bf10cded-60f7-47f8-800b-fb1b9b078776 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:57.080846 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bf10cded-60f7-47f8-800b-fb1b9b078776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:57.141585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1292/2572] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:57.149379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105af2f1-bd52-4bb9-9187-770a7813f4b1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:57.149668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105af2f1-bd52-4bb9-9187-770a7813f4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:57.149950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105af2f1-bd52-4bb9-9187-770a7813f4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:57.170212 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-105af2f1-bd52-4bb9-9187-770a7813f4b1 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:57.171458 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-105af2f1-bd52-4bb9-9187-770a7813f4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:57.179333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1281/2573] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:57.183536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10cded-60f7-47f8-800b-fb1b9b078776 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:57.193280 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10cded-60f7-47f8-800b-fb1b9b078776 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:57.197874 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b862c-706b-409d-bb69-4dffd19427cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:57.198474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b862c-706b-409d-bb69-4dffd19427cb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:57.198756 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b862c-706b-409d-bb69-4dffd19427cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:57.199029 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b862c-706b-409d-bb69-4dffd19427cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:57.199705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b862c-706b-409d-bb69-4dffd19427cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:21:57.200010 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b862c-706b-409d-bb69-4dffd19427cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:57.202096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b862c-706b-409d-bb69-4dffd19427cb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:57.213461 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1293/2574] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:21:57.218309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105af2f1-bd52-4bb9-9187-770a7813f4b1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:57.227551 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105af2f1-bd52-4bb9-9187-770a7813f4b1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:57.232291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a714f1-7519-40f7-9453-03551b52c099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:57.233159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a714f1-7519-40f7-9453-03551b52c099 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:57.233545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a714f1-7519-40f7-9453-03551b52c099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:57.233820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a714f1-7519-40f7-9453-03551b52c099 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:57.234476 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a714f1-7519-40f7-9453-03551b52c099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:57.236420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a714f1-7519-40f7-9453-03551b52c099 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:57.267151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1282/2575] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:57 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:21:57.271543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b862c-706b-409d-bb69-4dffd19427cb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:57.282122 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b862c-706b-409d-bb69-4dffd19427cb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:57.342827 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e96e4-63d8-45ed-a19c-0abe53b04cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:57.343485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e96e4-63d8-45ed-a19c-0abe53b04cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:57.343770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e96e4-63d8-45ed-a19c-0abe53b04cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:57.344048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e96e4-63d8-45ed-a19c-0abe53b04cc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:57.344687 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e96e4-63d8-45ed-a19c-0abe53b04cc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:57.346646 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e96e4-63d8-45ed-a19c-0abe53b04cc6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:57.357065 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:57.374983 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1294/2576] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:57 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:57.376280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a714f1-7519-40f7-9453-03551b52c099 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:57.386743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a714f1-7519-40f7-9453-03551b52c099 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:57.392164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155bd28-ba35-4454-a3fe-f6543f7c75dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:57.392847 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155bd28-ba35-4454-a3fe-f6543f7c75dc admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:57.393222 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155bd28-ba35-4454-a3fe-f6543f7c75dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:57.393562 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155bd28-ba35-4454-a3fe-f6543f7c75dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:57.394278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155bd28-ba35-4454-a3fe-f6543f7c75dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:57.396117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155bd28-ba35-4454-a3fe-f6543f7c75dc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:57.406857 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1283/2577] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:57.410830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e96e4-63d8-45ed-a19c-0abe53b04cc6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:57.421360 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e96e4-63d8-45ed-a19c-0abe53b04cc6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:57.426007 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d01c0-b102-48aa-943d-85297bd255fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:57.426608 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d01c0-b102-48aa-943d-85297bd255fb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:57.426891 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d01c0-b102-48aa-943d-85297bd255fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:57.427206 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d01c0-b102-48aa-943d-85297bd255fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:57.427882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d01c0-b102-48aa-943d-85297bd255fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:57.429487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d01c0-b102-48aa-943d-85297bd255fb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:57.441088 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1295/2578] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:57.445186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155bd28-ba35-4454-a3fe-f6543f7c75dc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:57.455913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155bd28-ba35-4454-a3fe-f6543f7c75dc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:57.460663 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c0796-3c2c-47f3-832c-50cd1d61e00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:57.461317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c0796-3c2c-47f3-832c-50cd1d61e00d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:57.461597 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c0796-3c2c-47f3-832c-50cd1d61e00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:57.461870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c0796-3c2c-47f3-832c-50cd1d61e00d admin admin] PATH_INFO: `/v3/projects/df44b24ebcae4760bc3f56fbee1cab00/users/9918579a1f88477cbbdedcca04bf0808/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:57.476511 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c0796-3c2c-47f3-832c-50cd1d61e00d admin admin] RBAC: Authorizing `identity:create_grant(project_id=df44b24ebcae4760bc3f56fbee1cab00, user_id=9918579a1f88477cbbdedcca04bf0808, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:57.479181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c0796-3c2c-47f3-832c-50cd1d61e00d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:57.492831 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1284/2579] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:57 2022] PUT /identity/v3/projects/df44b24ebcae4760bc3f56fbee1cab00/users/9918579a1f88477cbbdedcca04bf0808/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:57.498603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941036c-38b4-49cd-b432-96b3fac69026 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:57.499101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941036c-38b4-49cd-b432-96b3fac69026 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:57.499527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941036c-38b4-49cd-b432-96b3fac69026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:57.520397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7941036c-38b4-49cd-b432-96b3fac69026 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:57.521665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7941036c-38b4-49cd-b432-96b3fac69026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:57.706721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1296/2580] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:57.716058 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c0796-3c2c-47f3-832c-50cd1d61e00d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:57.718553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a73c0796-3c2c-47f3-832c-50cd1d61e00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:57.776659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c0796-3c2c-47f3-832c-50cd1d61e00d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:57.781181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8d5eb-3a50-4632-ab07-e0fc76d510b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:57.782069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8d5eb-3a50-4632-ab07-e0fc76d510b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:57.782422 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8d5eb-3a50-4632-ab07-e0fc76d510b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:57.782744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8d5eb-3a50-4632-ab07-e0fc76d510b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:57.792384 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8d5eb-3a50-4632-ab07-e0fc76d510b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:57.793809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8d5eb-3a50-4632-ab07-e0fc76d510b7 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:57.797650 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:57.804365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1285/2581] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:21:58.779822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b45dce-d7e7-4d9c-8829-c5a60c62df2c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:58.780324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b45dce-d7e7-4d9c-8829-c5a60c62df2c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:58.780667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b45dce-d7e7-4d9c-8829-c5a60c62df2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:58.812254 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-48b45dce-d7e7-4d9c-8829-c5a60c62df2c None None] MFA Rules not processed for user `9918579a1f88477cbbdedcca04bf0808`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:21:58.814837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-48b45dce-d7e7-4d9c-8829-c5a60c62df2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:21:58.895334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1297/2582] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:21:58 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:21:58.901333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8d5eb-3a50-4632-ab07-e0fc76d510b7 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:58.911352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8d5eb-3a50-4632-ab07-e0fc76d510b7 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:58.917746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fc8b9-2a1e-4ba4-9f5f-cd600fe3ffb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:58.918179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fc8b9-2a1e-4ba4-9f5f-cd600fe3ffb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:58.918333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fc8b9-2a1e-4ba4-9f5f-cd600fe3ffb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:58.918551 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fc8b9-2a1e-4ba4-9f5f-cd600fe3ffb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:58.919334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fc8b9-2a1e-4ba4-9f5f-cd600fe3ffb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:58.921787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fc8b9-2a1e-4ba4-9f5f-cd600fe3ffb9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:58.967104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1286/2583] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:21:58 2022] POST /identity/v3/projects => generated 403 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:21:58.970243 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b45dce-d7e7-4d9c-8829-c5a60c62df2c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:58.981351 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b45dce-d7e7-4d9c-8829-c5a60c62df2c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:59.037879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e94222-b23c-402e-8551-59a65da19100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:59.038484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e94222-b23c-402e-8551-59a65da19100 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:59.038761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e94222-b23c-402e-8551-59a65da19100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:59.039031 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e94222-b23c-402e-8551-59a65da19100 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:59.039693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e94222-b23c-402e-8551-59a65da19100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:59.041580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e94222-b23c-402e-8551-59a65da19100 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:59.067009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1298/2584] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:21:58 2022] POST /identity/v3/users => generated 308 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:21:59.068448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fc8b9-2a1e-4ba4-9f5f-cd600fe3ffb9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:59.079260 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fc8b9-2a1e-4ba4-9f5f-cd600fe3ffb9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:59.083923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a224d-c8b9-4b9b-8075-96e74ba5b223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:59.084535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a224d-c8b9-4b9b-8075-96e74ba5b223 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:59.084818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a224d-c8b9-4b9b-8075-96e74ba5b223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:59.085186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a224d-c8b9-4b9b-8075-96e74ba5b223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:59.085755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a224d-c8b9-4b9b-8075-96e74ba5b223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:59.087371 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a224d-c8b9-4b9b-8075-96e74ba5b223 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:59.098784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1287/2585] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:59.103023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e94222-b23c-402e-8551-59a65da19100 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:59.108729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:21:59.115232 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e94222-b23c-402e-8551-59a65da19100 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:59.120245 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90f8a8-5414-492a-8cec-f429a334098b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:59.120910 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90f8a8-5414-492a-8cec-f429a334098b admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:59.121212 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90f8a8-5414-492a-8cec-f429a334098b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:59.121479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90f8a8-5414-492a-8cec-f429a334098b admin admin] PATH_INFO: `/v3/projects/43a49fc15a6149f3a303a769d82bfac7/users/3f7a29aa008f4090b4e8c6ea14720cb7/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:59.139593 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90f8a8-5414-492a-8cec-f429a334098b admin admin] RBAC: Authorizing `identity:create_grant(project_id=43a49fc15a6149f3a303a769d82bfac7, user_id=3f7a29aa008f4090b4e8c6ea14720cb7, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:59.142226 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90f8a8-5414-492a-8cec-f429a334098b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:59.155859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1299/2586] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:59 2022] PUT /identity/v3/projects/43a49fc15a6149f3a303a769d82bfac7/users/3f7a29aa008f4090b4e8c6ea14720cb7/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:21:59.159868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a224d-c8b9-4b9b-8075-96e74ba5b223 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:59.170509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a224d-c8b9-4b9b-8075-96e74ba5b223 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:59.226023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f93ac-6ec2-4f2c-b9d3-4b58520c3432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:59.226568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f93ac-6ec2-4f2c-b9d3-4b58520c3432 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:59.226845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f93ac-6ec2-4f2c-b9d3-4b58520c3432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:59.227150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f93ac-6ec2-4f2c-b9d3-4b58520c3432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:59.227791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f93ac-6ec2-4f2c-b9d3-4b58520c3432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:59.229380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f93ac-6ec2-4f2c-b9d3-4b58520c3432 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:59.240074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1288/2587] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:21:59.244209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90f8a8-5414-492a-8cec-f429a334098b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:21:59.255147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90f8a8-5414-492a-8cec-f429a334098b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:21:59.259706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9985b-a0af-460e-b4e6-beafd6a1f14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:21:59.260343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9985b-a0af-460e-b4e6-beafd6a1f14c admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:21:59.260618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9985b-a0af-460e-b4e6-beafd6a1f14c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:21:59.260885 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9985b-a0af-460e-b4e6-beafd6a1f14c admin admin] PATH_INFO: `/v3/projects/43a49fc15a6149f3a303a769d82bfac7/users/3f7a29aa008f4090b4e8c6ea14720cb7/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:21:59.262778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9985b-a0af-460e-b4e6-beafd6a1f14c admin admin] RBAC: Authorizing `identity:create_grant(project_id=43a49fc15a6149f3a303a769d82bfac7, user_id=3f7a29aa008f4090b4e8c6ea14720cb7, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:21:59.265431 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9985b-a0af-460e-b4e6-beafd6a1f14c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:21:59.275665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1300/2588] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:21:59 2022] PUT /identity/v3/projects/43a49fc15a6149f3a303a769d82bfac7/users/3f7a29aa008f4090b4e8c6ea14720cb7/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:00.007394 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48da75c-79d4-4a75-9f8b-fd7874c480df None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:00.008281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48da75c-79d4-4a75-9f8b-fd7874c480df None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:00.008617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48da75c-79d4-4a75-9f8b-fd7874c480df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:00.042349 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e48da75c-79d4-4a75-9f8b-fd7874c480df None None] MFA Rules not processed for user `3f7a29aa008f4090b4e8c6ea14720cb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:00.043708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e48da75c-79d4-4a75-9f8b-fd7874c480df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:00.120255 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1289/2589] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:00 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:01.166204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9985b-a0af-460e-b4e6-beafd6a1f14c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:01.232061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9985b-a0af-460e-b4e6-beafd6a1f14c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:01.237779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:01.238479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:01.238821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:01.239203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] PATH_INFO: `/v3/users/9918579a1f88477cbbdedcca04bf0808` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:01.240399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9918579a1f88477cbbdedcca04bf0808)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:01.242368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:01.290986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9918579a1f88477cbbdedcca04bf0808'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.299841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9918579a1f88477cbbdedcca04bf0808'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.300477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9918579a1f88477cbbdedcca04bf0808'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.313169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9918579a1f88477cbbdedcca04bf0808'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.320840 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9918579a1f88477cbbdedcca04bf0808'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.327642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1301/2590] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:22:01 2022] DELETE /identity/v3/users/9918579a1f88477cbbdedcca04bf0808 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:01.500405 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48da75c-79d4-4a75-9f8b-fd7874c480df None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:01.511107 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48da75c-79d4-4a75-9f8b-fd7874c480df None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:01.556717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:01.570827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:01.571522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:01.571821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:01.572091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] PATH_INFO: `/v3/projects/df44b24ebcae4760bc3f56fbee1cab00` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:01.573234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] RBAC: Authorizing `identity:delete_project(project_id=df44b24ebcae4760bc3f56fbee1cab00)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:01.575152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:01.626142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df44b24ebcae4760bc3f56fbee1cab00', 'request_id': 'req-49d5552f-ecfe-46cf-b50c-e3690c9817be'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.631452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df44b24ebcae4760bc3f56fbee1cab00', 'request_id': 'req-49d5552f-ecfe-46cf-b50c-e3690c9817be'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.637671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df44b24ebcae4760bc3f56fbee1cab00', 'request_id': 'req-49d5552f-ecfe-46cf-b50c-e3690c9817be'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.650480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] The token cache is being invalidate because project df44b24ebcae4760bc3f56fbee1cab00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:22:01.650772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.651438 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.652985 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1290/2591] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:22:01 2022] DELETE /identity/v3/projects/df44b24ebcae4760bc3f56fbee1cab00 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:01.657901 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:01.659369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:01.672331 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db1b65-c83e-45d6-949e-8a66acd66a0c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:01.712598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:01.730391 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6398a1-d426-47a6-ba6c-5727b6e997a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:01.731027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6398a1-d426-47a6-ba6c-5727b6e997a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:01.731352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6398a1-d426-47a6-ba6c-5727b6e997a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:01.731624 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6398a1-d426-47a6-ba6c-5727b6e997a6 admin admin] PATH_INFO: `/v3/users/3f7a29aa008f4090b4e8c6ea14720cb7` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:01.732861 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6398a1-d426-47a6-ba6c-5727b6e997a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f7a29aa008f4090b4e8c6ea14720cb7)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:01.734783 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6398a1-d426-47a6-ba6c-5727b6e997a6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:01.780685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fd6398a1-d426-47a6-ba6c-5727b6e997a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f7a29aa008f4090b4e8c6ea14720cb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.787984 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fd6398a1-d426-47a6-ba6c-5727b6e997a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f7a29aa008f4090b4e8c6ea14720cb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.788588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fd6398a1-d426-47a6-ba6c-5727b6e997a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f7a29aa008f4090b4e8c6ea14720cb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.800374 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fd6398a1-d426-47a6-ba6c-5727b6e997a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f7a29aa008f4090b4e8c6ea14720cb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.807034 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fd6398a1-d426-47a6-ba6c-5727b6e997a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f7a29aa008f4090b4e8c6ea14720cb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:01.813715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1302/2592] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:22:01 2022] DELETE /identity/v3/users/3f7a29aa008f4090b4e8c6ea14720cb7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:01.990903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:01.992441 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:02.012969 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5552f-ecfe-46cf-b50c-e3690c9817be admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:02.072892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0fb46-4a21-4dbf-a79c-12b039db84b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:02.073539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0fb46-4a21-4dbf-a79c-12b039db84b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.073817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0fb46-4a21-4dbf-a79c-12b039db84b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.074160 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0fb46-4a21-4dbf-a79c-12b039db84b5 admin admin] PATH_INFO: `/v3/projects/43a49fc15a6149f3a303a769d82bfac7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.075340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0fb46-4a21-4dbf-a79c-12b039db84b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43a49fc15a6149f3a303a769d82bfac7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:02.079856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0fb46-4a21-4dbf-a79c-12b039db84b5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:02.129839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-86b0fb46-4a21-4dbf-a79c-12b039db84b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43a49fc15a6149f3a303a769d82bfac7', 'request_id': 'req-86b0fb46-4a21-4dbf-a79c-12b039db84b5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:02.135437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-86b0fb46-4a21-4dbf-a79c-12b039db84b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43a49fc15a6149f3a303a769d82bfac7', 'request_id': 'req-86b0fb46-4a21-4dbf-a79c-12b039db84b5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:02.141914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-86b0fb46-4a21-4dbf-a79c-12b039db84b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43a49fc15a6149f3a303a769d82bfac7', 'request_id': 'req-86b0fb46-4a21-4dbf-a79c-12b039db84b5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:02.154501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-86b0fb46-4a21-4dbf-a79c-12b039db84b5 admin admin] The token cache is being invalidate because project 43a49fc15a6149f3a303a769d82bfac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:22:02.154821 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-86b0fb46-4a21-4dbf-a79c-12b039db84b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:02.155468 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-86b0fb46-4a21-4dbf-a79c-12b039db84b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:02.157080 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1291/2593] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:22:01 2022] DELETE /identity/v3/projects/43a49fc15a6149f3a303a769d82bfac7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:02.161800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8f397-e5c7-4960-b337-81b2052d4c47 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.162602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8f397-e5c7-4960-b337-81b2052d4c47 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.162602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8f397-e5c7-4960-b337-81b2052d4c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.183546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8ad8f397-e5c7-4960-b337-81b2052d4c47 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:02.184758 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8ad8f397-e5c7-4960-b337-81b2052d4c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:02.246308 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1303/2594] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:02.255080 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b3e6d-5a9c-4079-a73b-f9c62e8c5a1c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.255471 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b3e6d-5a9c-4079-a73b-f9c62e8c5a1c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.255632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b3e6d-5a9c-4079-a73b-f9c62e8c5a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.269609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:02.276970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-600b3e6d-5a9c-4079-a73b-f9c62e8c5a1c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:02.278768 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-600b3e6d-5a9c-4079-a73b-f9c62e8c5a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:02.288192 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1292/2595] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:02.292874 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8f397-e5c7-4960-b337-81b2052d4c47 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:02.303402 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8f397-e5c7-4960-b337-81b2052d4c47 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:02.308061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7c0c1-cc6a-4944-a7bf-db317e878f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:02.308664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7c0c1-cc6a-4944-a7bf-db317e878f49 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.308935 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7c0c1-cc6a-4944-a7bf-db317e878f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.309199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7c0c1-cc6a-4944-a7bf-db317e878f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.309864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7c0c1-cc6a-4944-a7bf-db317e878f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:22:02.310273 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7c0c1-cc6a-4944-a7bf-db317e878f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:02.312895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7c0c1-cc6a-4944-a7bf-db317e878f49 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:02.328098 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1304/2596] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:22:02.332344 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b3e6d-5a9c-4079-a73b-f9c62e8c5a1c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:02.341562 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b3e6d-5a9c-4079-a73b-f9c62e8c5a1c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:02.346292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c8779-fb63-4b0c-8536-c75d47d3fbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:02.346935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c8779-fb63-4b0c-8536-c75d47d3fbb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.347252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c8779-fb63-4b0c-8536-c75d47d3fbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.347523 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c8779-fb63-4b0c-8536-c75d47d3fbb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.348144 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c8779-fb63-4b0c-8536-c75d47d3fbb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:02.350025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c8779-fb63-4b0c-8536-c75d47d3fbb8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:02.378285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1293/2597] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:22:02 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:02.382520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7c0c1-cc6a-4944-a7bf-db317e878f49 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:02.392733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7c0c1-cc6a-4944-a7bf-db317e878f49 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:02.443345 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:02.455146 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea83a7-a499-4996-b64b-c2ef56e852d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:02.455765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea83a7-a499-4996-b64b-c2ef56e852d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.456048 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea83a7-a499-4996-b64b-c2ef56e852d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.456321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea83a7-a499-4996-b64b-c2ef56e852d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.456953 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea83a7-a499-4996-b64b-c2ef56e852d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:02.458864 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea83a7-a499-4996-b64b-c2ef56e852d3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:02.483113 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1305/2598] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:22:02 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:02.487355 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c8779-fb63-4b0c-8536-c75d47d3fbb8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:02.497561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c8779-fb63-4b0c-8536-c75d47d3fbb8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:02.502655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c3527-482f-4e11-9e56-269d8e256c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:02.503304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c3527-482f-4e11-9e56-269d8e256c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.503564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c3527-482f-4e11-9e56-269d8e256c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.503816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c3527-482f-4e11-9e56-269d8e256c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.504453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c3527-482f-4e11-9e56-269d8e256c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:02.506155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c3527-482f-4e11-9e56-269d8e256c7d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:02.517551 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1294/2599] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:22:02.521139 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea83a7-a499-4996-b64b-c2ef56e852d3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:02.532109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea83a7-a499-4996-b64b-c2ef56e852d3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:02.541509 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d70c6-0ff3-4aa5-9cf8-4489a1164a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:02.542119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d70c6-0ff3-4aa5-9cf8-4489a1164a62 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.542400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d70c6-0ff3-4aa5-9cf8-4489a1164a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.542673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d70c6-0ff3-4aa5-9cf8-4489a1164a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.543343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d70c6-0ff3-4aa5-9cf8-4489a1164a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:02.545036 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d70c6-0ff3-4aa5-9cf8-4489a1164a62 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:02.556376 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1306/2600] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:22:02.560483 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c3527-482f-4e11-9e56-269d8e256c7d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:02.570460 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c3527-482f-4e11-9e56-269d8e256c7d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:02.575068 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24ede7-a85d-46aa-a728-eac3f998e543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:02.575729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24ede7-a85d-46aa-a728-eac3f998e543 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.576012 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24ede7-a85d-46aa-a728-eac3f998e543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.576299 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24ede7-a85d-46aa-a728-eac3f998e543 admin admin] PATH_INFO: `/v3/projects/28ad53642ea841e39602cddb7cffc571/users/172b5d2bbb3545b8a6f7dac85d55470e/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.591300 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24ede7-a85d-46aa-a728-eac3f998e543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28ad53642ea841e39602cddb7cffc571, user_id=172b5d2bbb3545b8a6f7dac85d55470e, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:02.594575 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24ede7-a85d-46aa-a728-eac3f998e543 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:02.608462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1295/2601] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:02 2022] PUT /identity/v3/projects/28ad53642ea841e39602cddb7cffc571/users/172b5d2bbb3545b8a6f7dac85d55470e/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:02.618881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760579a1-6548-4221-951c-23a83fff96ff None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.619201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760579a1-6548-4221-951c-23a83fff96ff None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.619506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760579a1-6548-4221-951c-23a83fff96ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.647462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-760579a1-6548-4221-951c-23a83fff96ff None None] MFA Rules not processed for user `172b5d2bbb3545b8a6f7dac85d55470e`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:02.648672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-760579a1-6548-4221-951c-23a83fff96ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:02.712590 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1307/2602] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:02.717316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24ede7-a85d-46aa-a728-eac3f998e543 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:02.781207 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24ede7-a85d-46aa-a728-eac3f998e543 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:02.785950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9529a-076f-4f55-bf59-ccc87053302c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:02.786561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9529a-076f-4f55-bf59-ccc87053302c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.786839 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9529a-076f-4f55-bf59-ccc87053302c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.787141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9529a-076f-4f55-bf59-ccc87053302c admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.787786 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9529a-076f-4f55-bf59-ccc87053302c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:02.789697 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9529a-076f-4f55-bf59-ccc87053302c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:02.819717 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1296/2603] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:22:02 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:02.822536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760579a1-6548-4221-951c-23a83fff96ff None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:02.834283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760579a1-6548-4221-951c-23a83fff96ff None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:02.892177 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89fad10-9c9c-4466-9e8b-4e7d01aa4c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:02.892797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89fad10-9c9c-4466-9e8b-4e7d01aa4c7b admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.893079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89fad10-9c9c-4466-9e8b-4e7d01aa4c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.893376 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89fad10-9c9c-4466-9e8b-4e7d01aa4c7b admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.894015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89fad10-9c9c-4466-9e8b-4e7d01aa4c7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:02.896075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89fad10-9c9c-4466-9e8b-4e7d01aa4c7b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:02.918353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1308/2604] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:22:02 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:02.922587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9529a-076f-4f55-bf59-ccc87053302c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:02.932977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9529a-076f-4f55-bf59-ccc87053302c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:02.937858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3865b6-19ee-41b1-815c-dd4ef9f157aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:02.938466 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3865b6-19ee-41b1-815c-dd4ef9f157aa admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.938748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3865b6-19ee-41b1-815c-dd4ef9f157aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.939025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3865b6-19ee-41b1-815c-dd4ef9f157aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.939704 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3865b6-19ee-41b1-815c-dd4ef9f157aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:02.941305 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3865b6-19ee-41b1-815c-dd4ef9f157aa admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:02.952530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1297/2605] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:22:02.956719 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89fad10-9c9c-4466-9e8b-4e7d01aa4c7b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:02.966645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89fad10-9c9c-4466-9e8b-4e7d01aa4c7b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:02.971790 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf9bb4-e36e-4127-9c8c-4d7a99801dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:02.972450 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf9bb4-e36e-4127-9c8c-4d7a99801dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:02.972729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf9bb4-e36e-4127-9c8c-4d7a99801dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:02.973055 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf9bb4-e36e-4127-9c8c-4d7a99801dcf admin admin] PATH_INFO: `/v3/projects/b767efa493f8446f9fa5d4332d4ccabe/users/f68c07c5a0de4e88820c3cc818d25a6b/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:02.988137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf9bb4-e36e-4127-9c8c-4d7a99801dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b767efa493f8446f9fa5d4332d4ccabe, user_id=f68c07c5a0de4e88820c3cc818d25a6b, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:02.990755 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf9bb4-e36e-4127-9c8c-4d7a99801dcf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:03.005543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1309/2606] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:02 2022] PUT /identity/v3/projects/b767efa493f8446f9fa5d4332d4ccabe/users/f68c07c5a0de4e88820c3cc818d25a6b/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:03.009489 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3865b6-19ee-41b1-815c-dd4ef9f157aa admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:03.020743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3865b6-19ee-41b1-815c-dd4ef9f157aa admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:03.078870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e32bfe-1673-4475-b745-aa9de0a7583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:03.079529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e32bfe-1673-4475-b745-aa9de0a7583f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:03.079808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e32bfe-1673-4475-b745-aa9de0a7583f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:03.080082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e32bfe-1673-4475-b745-aa9de0a7583f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:03.080723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e32bfe-1673-4475-b745-aa9de0a7583f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:03.082334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e32bfe-1673-4475-b745-aa9de0a7583f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:03.097446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1298/2607] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:22:03.099220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf9bb4-e36e-4127-9c8c-4d7a99801dcf admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:03.111680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf9bb4-e36e-4127-9c8c-4d7a99801dcf admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:03.116498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f93394-8d7a-4e10-8dfd-25aef01bc98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:03.117159 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f93394-8d7a-4e10-8dfd-25aef01bc98a admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:03.117439 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f93394-8d7a-4e10-8dfd-25aef01bc98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:03.117713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f93394-8d7a-4e10-8dfd-25aef01bc98a admin admin] PATH_INFO: `/v3/projects/b767efa493f8446f9fa5d4332d4ccabe/users/f68c07c5a0de4e88820c3cc818d25a6b/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:03.119900 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f93394-8d7a-4e10-8dfd-25aef01bc98a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b767efa493f8446f9fa5d4332d4ccabe, user_id=f68c07c5a0de4e88820c3cc818d25a6b, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:03.122530 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f93394-8d7a-4e10-8dfd-25aef01bc98a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:03.133758 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1310/2608] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:03 2022] PUT /identity/v3/projects/b767efa493f8446f9fa5d4332d4ccabe/users/f68c07c5a0de4e88820c3cc818d25a6b/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:03.145745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b2176-80e7-4db2-8cf6-d44ddedc4e39 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:03.146047 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b2176-80e7-4db2-8cf6-d44ddedc4e39 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:03.146732 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b2176-80e7-4db2-8cf6-d44ddedc4e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:03.174770 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:03.178298 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-400b2176-80e7-4db2-8cf6-d44ddedc4e39 None None] MFA Rules not processed for user `f68c07c5a0de4e88820c3cc818d25a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:03.180448 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-400b2176-80e7-4db2-8cf6-d44ddedc4e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:03.273605 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1299/2609] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:03.286022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f93394-8d7a-4e10-8dfd-25aef01bc98a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:03.288460 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-67f93394-8d7a-4e10-8dfd-25aef01bc98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:03.303079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:03.350068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f93394-8d7a-4e10-8dfd-25aef01bc98a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:03.353789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2d0ff-5fc0-41ca-9428-d8fe62f8a861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:03.354396 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2d0ff-5fc0-41ca-9428-d8fe62f8a861 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:03.354672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2d0ff-5fc0-41ca-9428-d8fe62f8a861 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:03.354945 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2d0ff-5fc0-41ca-9428-d8fe62f8a861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:03.365002 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2d0ff-5fc0-41ca-9428-d8fe62f8a861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:03.366426 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2d0ff-5fc0-41ca-9428-d8fe62f8a861 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:03.376932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1311/2610] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:03.701748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b2176-80e7-4db2-8cf6-d44ddedc4e39 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:03.711280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b2176-80e7-4db2-8cf6-d44ddedc4e39 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:03.716189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a7644-3332-4d9f-be01-141b8ede82f8 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f68c07c5a0de4e88820c3cc818d25a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b767efa493f8446f9fa5d4332d4ccabe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:03.717773 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a7644-3332-4d9f-be01-141b8ede82f8 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:03.718111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a7644-3332-4d9f-be01-141b8ede82f8 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:03.718390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a7644-3332-4d9f-be01-141b8ede82f8 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:03.719044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a7644-3332-4d9f-be01-141b8ede82f8 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:22:03.719383 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a7644-3332-4d9f-be01-141b8ede82f8 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:03.720986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a7644-3332-4d9f-be01-141b8ede82f8 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:03.733398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1300/2611] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:22:03.737651 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2d0ff-5fc0-41ca-9428-d8fe62f8a861 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:03.746768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2d0ff-5fc0-41ca-9428-d8fe62f8a861 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:03.751920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833d2d2-366f-4108-9a84-96a3528a12e2 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f68c07c5a0de4e88820c3cc818d25a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b767efa493f8446f9fa5d4332d4ccabe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:03.752505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833d2d2-366f-4108-9a84-96a3528a12e2 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:03.752788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833d2d2-366f-4108-9a84-96a3528a12e2 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:03.753060 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833d2d2-366f-4108-9a84-96a3528a12e2 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:03.753726 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833d2d2-366f-4108-9a84-96a3528a12e2 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:03.755620 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833d2d2-366f-4108-9a84-96a3528a12e2 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:03.783017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1312/2612] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:22:03 2022] POST /identity/v3/projects => generated 354 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:03.787236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a7644-3332-4d9f-be01-141b8ede82f8 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:03.797342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a7644-3332-4d9f-be01-141b8ede82f8 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:03.854137 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4dd7d-498e-4846-a6a3-1160763fe730 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f68c07c5a0de4e88820c3cc818d25a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b767efa493f8446f9fa5d4332d4ccabe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:03.854748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4dd7d-498e-4846-a6a3-1160763fe730 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:03.855086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4dd7d-498e-4846-a6a3-1160763fe730 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:03.855388 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4dd7d-498e-4846-a6a3-1160763fe730 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:03.856024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4dd7d-498e-4846-a6a3-1160763fe730 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:22:03.856347 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4dd7d-498e-4846-a6a3-1160763fe730 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:03.858082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4dd7d-498e-4846-a6a3-1160763fe730 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:03.869741 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1301/2613] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:22:03.907430 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833d2d2-366f-4108-9a84-96a3528a12e2 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:03.917788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833d2d2-366f-4108-9a84-96a3528a12e2 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:03.922622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f68c07c5a0de4e88820c3cc818d25a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b767efa493f8446f9fa5d4332d4ccabe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:03.923344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:03.923622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:03.923892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] PATH_INFO: `/v3/projects/a17172eb00df41f19307ce6763e21431` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:03.925078 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a17172eb00df41f19307ce6763e21431)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:03.926952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:03.975141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a17172eb00df41f19307ce6763e21431', 'request_id': 'req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:03.982616 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a17172eb00df41f19307ce6763e21431', 'request_id': 'req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:03.988642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a17172eb00df41f19307ce6763e21431', 'request_id': 'req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:04.001240 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] The token cache is being invalidate because project a17172eb00df41f19307ce6763e21431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:22:04.001520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:04.002157 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-be9e0e86-98ec-4d1b-90ad-ff3adb39de70 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:04.003644 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1313/2614] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:22:03 2022] DELETE /identity/v3/projects/a17172eb00df41f19307ce6763e21431 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:04.204477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4dd7d-498e-4846-a6a3-1160763fe730 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:04.206225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1dd4dd7d-498e-4846-a6a3-1160763fe730 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:04.270163 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4dd7d-498e-4846-a6a3-1160763fe730 tempest-VolumeQuotaClassesTest-1244444030 tempest-VolumeQuotaClassesTest-1244444030-project-admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:04.293126 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:04.327764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:04.328324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:04.328606 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:04.328883 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] PATH_INFO: `/v3/users/172b5d2bbb3545b8a6f7dac85d55470e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:04.330020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=172b5d2bbb3545b8a6f7dac85d55470e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:04.332002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:04.379035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172b5d2bbb3545b8a6f7dac85d55470e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:04.386737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172b5d2bbb3545b8a6f7dac85d55470e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:04.399182 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172b5d2bbb3545b8a6f7dac85d55470e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:04.399830 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172b5d2bbb3545b8a6f7dac85d55470e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:04.406970 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172b5d2bbb3545b8a6f7dac85d55470e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:04.412868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1302/2615] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:22:04 2022] DELETE /identity/v3/users/172b5d2bbb3545b8a6f7dac85d55470e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:04.417361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0561405-3ae3-410c-9d31-45149a50d288 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:04.417814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0561405-3ae3-410c-9d31-45149a50d288 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:04.417934 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0561405-3ae3-410c-9d31-45149a50d288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:04.437841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e0561405-3ae3-410c-9d31-45149a50d288 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:04.439033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e0561405-3ae3-410c-9d31-45149a50d288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:04.496677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1314/2616] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:04.507390 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:04.508834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:04.578063 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698743d6-0634-43ea-a8fc-bfc263ad7e06 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:04.582913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207df296-20cc-4d40-ae30-6e2b17c177b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:04.583756 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207df296-20cc-4d40-ae30-6e2b17c177b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:04.584216 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207df296-20cc-4d40-ae30-6e2b17c177b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:04.584643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207df296-20cc-4d40-ae30-6e2b17c177b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:04.596417 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207df296-20cc-4d40-ae30-6e2b17c177b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:04.598130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207df296-20cc-4d40-ae30-6e2b17c177b3 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:04.610643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1303/2617] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:22:05.060633 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0561405-3ae3-410c-9d31-45149a50d288 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:05.062704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e0561405-3ae3-410c-9d31-45149a50d288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:05.075260 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0561405-3ae3-410c-9d31-45149a50d288 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:05.080721 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:05.081363 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:05.081638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:05.081911 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] PATH_INFO: `/v3/projects/28ad53642ea841e39602cddb7cffc571` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:05.083120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28ad53642ea841e39602cddb7cffc571)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:05.085145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:05.097883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:05.136895 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28ad53642ea841e39602cddb7cffc571', 'request_id': 'req-947e680c-9f79-480a-8a64-1c24ba371304'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.144801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28ad53642ea841e39602cddb7cffc571', 'request_id': 'req-947e680c-9f79-480a-8a64-1c24ba371304'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.150322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28ad53642ea841e39602cddb7cffc571', 'request_id': 'req-947e680c-9f79-480a-8a64-1c24ba371304'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.162282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] The token cache is being invalidate because project 28ad53642ea841e39602cddb7cffc571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:22:05.162567 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.163253 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.164729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1315/2618] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:22:05 2022] DELETE /identity/v3/projects/28ad53642ea841e39602cddb7cffc571 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:05.168746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207df296-20cc-4d40-ae30-6e2b17c177b3 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:05.178662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207df296-20cc-4d40-ae30-6e2b17c177b3 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:05.223828 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:05.237909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf24d3-fdf8-446d-9378-6ade0a052f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:05.238542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf24d3-fdf8-446d-9378-6ade0a052f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:05.238819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf24d3-fdf8-446d-9378-6ade0a052f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:05.239129 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf24d3-fdf8-446d-9378-6ade0a052f07 admin admin] PATH_INFO: `/v3/users/f68c07c5a0de4e88820c3cc818d25a6b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:05.240667 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf24d3-fdf8-446d-9378-6ade0a052f07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f68c07c5a0de4e88820c3cc818d25a6b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:05.242175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf24d3-fdf8-446d-9378-6ade0a052f07 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:05.293862 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-acaf24d3-fdf8-446d-9378-6ade0a052f07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f68c07c5a0de4e88820c3cc818d25a6b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.303162 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-acaf24d3-fdf8-446d-9378-6ade0a052f07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f68c07c5a0de4e88820c3cc818d25a6b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.317577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-acaf24d3-fdf8-446d-9378-6ade0a052f07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f68c07c5a0de4e88820c3cc818d25a6b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.318743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-acaf24d3-fdf8-446d-9378-6ade0a052f07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f68c07c5a0de4e88820c3cc818d25a6b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.328317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-acaf24d3-fdf8-446d-9378-6ade0a052f07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f68c07c5a0de4e88820c3cc818d25a6b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.335535 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1304/2619] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:22:05 2022] DELETE /identity/v3/users/f68c07c5a0de4e88820c3cc818d25a6b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:05.618844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:05.620296 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:05.634805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e680c-9f79-480a-8a64-1c24ba371304 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:05.692765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:05.693398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:05.693680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:05.693956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf admin admin] PATH_INFO: `/v3/projects/b767efa493f8446f9fa5d4332d4ccabe` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:05.695134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b767efa493f8446f9fa5d4332d4ccabe)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:05.697047 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:05.742348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b767efa493f8446f9fa5d4332d4ccabe', 'request_id': 'req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.750314 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b767efa493f8446f9fa5d4332d4ccabe', 'request_id': 'req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.756062 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b767efa493f8446f9fa5d4332d4ccabe', 'request_id': 'req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.770580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf admin admin] The token cache is being invalidate because project b767efa493f8446f9fa5d4332d4ccabe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:22:05.770896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.771742 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-32c8a133-5b2f-4acb-a98f-dd887eaa7faf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:05.773178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1316/2620] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:22:05 2022] DELETE /identity/v3/projects/b767efa493f8446f9fa5d4332d4ccabe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:05.778143 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8d2da-9c80-4c0e-b1df-4552e86fe189 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:05.778453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8d2da-9c80-4c0e-b1df-4552e86fe189 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:05.779616 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8d2da-9c80-4c0e-b1df-4552e86fe189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:05.812633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f7e8d2da-9c80-4c0e-b1df-4552e86fe189 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:05.813958 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f7e8d2da-9c80-4c0e-b1df-4552e86fe189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:05.879676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1305/2621] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:05.887167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169f523-7947-42e6-8ccf-932ecba0c16c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:05.887447 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169f523-7947-42e6-8ccf-932ecba0c16c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:05.887715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169f523-7947-42e6-8ccf-932ecba0c16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:05.906908 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6169f523-7947-42e6-8ccf-932ecba0c16c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:05.908148 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6169f523-7947-42e6-8ccf-932ecba0c16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:05.916022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1317/2622] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:05.920943 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8d2da-9c80-4c0e-b1df-4552e86fe189 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:05.933290 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8d2da-9c80-4c0e-b1df-4552e86fe189 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:05.941937 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa7471-18a8-4854-9dac-73c8cc863f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:05.942938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa7471-18a8-4854-9dac-73c8cc863f69 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:05.943233 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa7471-18a8-4854-9dac-73c8cc863f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:05.943533 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa7471-18a8-4854-9dac-73c8cc863f69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:05.944201 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa7471-18a8-4854-9dac-73c8cc863f69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:22:05.944517 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa7471-18a8-4854-9dac-73c8cc863f69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:05.946140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa7471-18a8-4854-9dac-73c8cc863f69 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:05.960993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1306/2623] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:22:05.965482 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169f523-7947-42e6-8ccf-932ecba0c16c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:05.976390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169f523-7947-42e6-8ccf-932ecba0c16c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:05.981124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1250a-bf1a-4598-9470-059428ffbc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:05.981729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1250a-bf1a-4598-9470-059428ffbc68 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:05.982129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1250a-bf1a-4598-9470-059428ffbc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:05.982975 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1250a-bf1a-4598-9470-059428ffbc68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:05.983718 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1250a-bf1a-4598-9470-059428ffbc68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:05.985597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1250a-bf1a-4598-9470-059428ffbc68 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:05.991545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa7471-18a8-4854-9dac-73c8cc863f69 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:05.993291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-69aa7471-18a8-4854-9dac-73c8cc863f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:06.009807 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:06.018996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1318/2624] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:22:05 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:06.025607 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1250a-bf1a-4598-9470-059428ffbc68 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:06.077020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa7471-18a8-4854-9dac-73c8cc863f69 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:06.091991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:06.105326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1250a-bf1a-4598-9470-059428ffbc68 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:06.111932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d529df-d66c-45f2-b17e-c0d86c52de91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:06.115690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d529df-d66c-45f2-b17e-c0d86c52de91 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:06.116077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d529df-d66c-45f2-b17e-c0d86c52de91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:06.116450 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d529df-d66c-45f2-b17e-c0d86c52de91 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:06.117498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d529df-d66c-45f2-b17e-c0d86c52de91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:06.119554 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d529df-d66c-45f2-b17e-c0d86c52de91 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:06.155679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1319/2625] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:22:06 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:22:06.156668 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d529df-d66c-45f2-b17e-c0d86c52de91 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:06.160228 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d684e-e78d-41c6-b9d6-8f639944d4fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:06.160861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d684e-e78d-41c6-b9d6-8f639944d4fc service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:06.161141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d684e-e78d-41c6-b9d6-8f639944d4fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:06.161415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d684e-e78d-41c6-b9d6-8f639944d4fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:06.163345 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-259d684e-e78d-41c6-b9d6-8f639944d4fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:06.168039 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d529df-d66c-45f2-b17e-c0d86c52de91 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:06.173851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ac209-f51b-437e-917b-4cd9c486f8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:06.174603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ac209-f51b-437e-917b-4cd9c486f8ad admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:06.174981 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ac209-f51b-437e-917b-4cd9c486f8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:06.175143 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d684e-e78d-41c6-b9d6-8f639944d4fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:06.175577 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ac209-f51b-437e-917b-4cd9c486f8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:06.176247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ac209-f51b-437e-917b-4cd9c486f8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:06.177044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d684e-e78d-41c6-b9d6-8f639944d4fc service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:06.178058 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ac209-f51b-437e-917b-4cd9c486f8ad admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:06.203238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1320/2626] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:22:06.204308 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ac209-f51b-437e-917b-4cd9c486f8ad admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:06.210081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1307/2627] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:22:06.219837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ac209-f51b-437e-917b-4cd9c486f8ad admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:06.227295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b2e6b-ccf9-4bec-af38-453d6fdcbc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:06.228327 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b2e6b-ccf9-4bec-af38-453d6fdcbc1c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:06.228680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b2e6b-ccf9-4bec-af38-453d6fdcbc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:06.229265 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b2e6b-ccf9-4bec-af38-453d6fdcbc1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:06.230147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b2e6b-ccf9-4bec-af38-453d6fdcbc1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:06.232475 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b2e6b-ccf9-4bec-af38-453d6fdcbc1c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:06.254471 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1321/2628] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:22:06.255776 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d684e-e78d-41c6-b9d6-8f639944d4fc service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:06.272415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d684e-e78d-41c6-b9d6-8f639944d4fc service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:06.277585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b053afe-0fa0-4646-bc62-a0b63d29beef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:06.278408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b053afe-0fa0-4646-bc62-a0b63d29beef admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:06.278687 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b053afe-0fa0-4646-bc62-a0b63d29beef admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:06.279011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b053afe-0fa0-4646-bc62-a0b63d29beef admin admin] PATH_INFO: `/v3/projects/30a4737358b2428e93b170ba4b4e17d5/users/e2279200063e46b6b017b9447900d57c/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:06.303961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b053afe-0fa0-4646-bc62-a0b63d29beef admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a4737358b2428e93b170ba4b4e17d5, user_id=e2279200063e46b6b017b9447900d57c, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:06.306673 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b053afe-0fa0-4646-bc62-a0b63d29beef admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:06.335195 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1308/2629] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:06 2022] PUT /identity/v3/projects/30a4737358b2428e93b170ba4b4e17d5/users/e2279200063e46b6b017b9447900d57c/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:06.344971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e80ea-bf0d-49a2-922c-079feb85bbbb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:06.345631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e80ea-bf0d-49a2-922c-079feb85bbbb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:06.345631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e80ea-bf0d-49a2-922c-079feb85bbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:06.401009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1c3e80ea-bf0d-49a2-922c-079feb85bbbb None None] MFA Rules not processed for user `e2279200063e46b6b017b9447900d57c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:06.402825 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1c3e80ea-bf0d-49a2-922c-079feb85bbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:06.518151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1322/2630] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:06.531127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b053afe-0fa0-4646-bc62-a0b63d29beef admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:06.533309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9b053afe-0fa0-4646-bc62-a0b63d29beef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:06.628480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b053afe-0fa0-4646-bc62-a0b63d29beef admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:06.633541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79e750-806c-488d-a9cc-34de071bd6a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:06.633935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79e750-806c-488d-a9cc-34de071bd6a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:06.634219 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79e750-806c-488d-a9cc-34de071bd6a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:06.634583 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79e750-806c-488d-a9cc-34de071bd6a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:06.645262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79e750-806c-488d-a9cc-34de071bd6a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:06.647569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79e750-806c-488d-a9cc-34de071bd6a7 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:06.658314 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1309/2631] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:06.681317 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e80ea-bf0d-49a2-922c-079feb85bbbb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:06.772310 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:06.778116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e80ea-bf0d-49a2-922c-079feb85bbbb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:06.784325 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:06.785301 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:06.785611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:06.785841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] PATH_INFO: `/v3/users/e2279200063e46b6b017b9447900d57c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:06.787308 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2279200063e46b6b017b9447900d57c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:06.789473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:06.851123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2279200063e46b6b017b9447900d57c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:06.861459 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2279200063e46b6b017b9447900d57c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:06.862199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2279200063e46b6b017b9447900d57c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:06.877250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2279200063e46b6b017b9447900d57c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:06.885076 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2279200063e46b6b017b9447900d57c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:06.891740 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1323/2632] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:22:06 2022] DELETE /identity/v3/users/e2279200063e46b6b017b9447900d57c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:06.896093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7bbbb-9743-4536-86cf-33affc79cc1f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:06.896380 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7bbbb-9743-4536-86cf-33affc79cc1f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:06.896652 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7bbbb-9743-4536-86cf-33affc79cc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:06.916983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f1a7bbbb-9743-4536-86cf-33affc79cc1f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:06.918218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f1a7bbbb-9743-4536-86cf-33affc79cc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:06.986108 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1310/2633] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:06.997361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:06.999271 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:07.061239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d322c0-61ac-476d-b6ae-ded4b41ab00b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:07.065067 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddb36a-ef80-4ae7-886f-97707fdd3107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:07.065779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddb36a-ef80-4ae7-886f-97707fdd3107 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:07.066060 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddb36a-ef80-4ae7-886f-97707fdd3107 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:07.066445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddb36a-ef80-4ae7-886f-97707fdd3107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:07.075740 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddb36a-ef80-4ae7-886f-97707fdd3107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:07.077293 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddb36a-ef80-4ae7-886f-97707fdd3107 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:07.087977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1324/2634] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:07.383755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7bbbb-9743-4536-86cf-33affc79cc1f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:07.385278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f1a7bbbb-9743-4536-86cf-33affc79cc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:07.398194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7bbbb-9743-4536-86cf-33affc79cc1f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:07.402869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:07.403537 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:07.403817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:07.404268 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] PATH_INFO: `/v3/projects/30a4737358b2428e93b170ba4b4e17d5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:07.405896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30a4737358b2428e93b170ba4b4e17d5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:07.407831 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:07.440990 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:07.460062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30a4737358b2428e93b170ba4b4e17d5', 'request_id': 'req-6b59bbc2-6d04-4acc-951a-131f509ac6b6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:07.465658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30a4737358b2428e93b170ba4b4e17d5', 'request_id': 'req-6b59bbc2-6d04-4acc-951a-131f509ac6b6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:07.472977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30a4737358b2428e93b170ba4b4e17d5', 'request_id': 'req-6b59bbc2-6d04-4acc-951a-131f509ac6b6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:07.486432 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] The token cache is being invalidate because project 30a4737358b2428e93b170ba4b4e17d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:22:07.486791 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:07.487453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:07.488933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1311/2635] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:22:07 2022] DELETE /identity/v3/projects/30a4737358b2428e93b170ba4b4e17d5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:17.082676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:17.209326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d666b43-1da7-45ff-90e2-7035ea01e860 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:17.209814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d666b43-1da7-45ff-90e2-7035ea01e860 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:17.210307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d666b43-1da7-45ff-90e2-7035ea01e860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:17.233170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7d666b43-1da7-45ff-90e2-7035ea01e860 None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:17.235242 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7d666b43-1da7-45ff-90e2-7035ea01e860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:17.297169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1325/2636] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:22:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:17.311678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:17.313859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:17.409288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59bbc2-6d04-4acc-951a-131f509ac6b6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:17.414526 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d13f0f-7627-49a5-b561-4aa1c6e09879 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:17.415252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d13f0f-7627-49a5-b561-4aa1c6e09879 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:17.415599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d13f0f-7627-49a5-b561-4aa1c6e09879 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:17.415939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d13f0f-7627-49a5-b561-4aa1c6e09879 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:17.425173 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d13f0f-7627-49a5-b561-4aa1c6e09879 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:17.426788 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d13f0f-7627-49a5-b561-4aa1c6e09879 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:17.435302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1312/2637] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:22:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:22:24.586235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d666b43-1da7-45ff-90e2-7035ea01e860 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:24.587973 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7d666b43-1da7-45ff-90e2-7035ea01e860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:24.658389 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d666b43-1da7-45ff-90e2-7035ea01e860 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:24.662572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b6044-880a-4392-b243-cf7dcd76c4c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:24.663241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b6044-880a-4392-b243-cf7dcd76c4c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:24.663526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b6044-880a-4392-b243-cf7dcd76c4c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:24.663798 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b6044-880a-4392-b243-cf7dcd76c4c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:24.665899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6a2b6044-880a-4392-b243-cf7dcd76c4c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:24.742420 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b6044-880a-4392-b243-cf7dcd76c4c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:24.743922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b6044-880a-4392-b243-cf7dcd76c4c4 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:24.755589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1326/2638] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:27.431819 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=46,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:28.635340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d13f0f-7627-49a5-b561-4aa1c6e09879 service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:28.637537 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-72d13f0f-7627-49a5-b561-4aa1c6e09879 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:28.654076 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d13f0f-7627-49a5-b561-4aa1c6e09879 service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:28.659496 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6a09e-d909-4b8f-96ec-8470267bf343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:28.660240 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6a09e-d909-4b8f-96ec-8470267bf343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:28.660529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6a09e-d909-4b8f-96ec-8470267bf343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:28.660852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6a09e-d909-4b8f-96ec-8470267bf343 admin admin] PATH_INFO: `/v3/users/dcb88983a8e1439588644b71170d7362` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:28.662178 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6a09e-d909-4b8f-96ec-8470267bf343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcb88983a8e1439588644b71170d7362)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:28.664204 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6a09e-d909-4b8f-96ec-8470267bf343 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:28.711711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e7c6a09e-d909-4b8f-96ec-8470267bf343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcb88983a8e1439588644b71170d7362'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:28.719130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e7c6a09e-d909-4b8f-96ec-8470267bf343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcb88983a8e1439588644b71170d7362'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:28.731123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e7c6a09e-d909-4b8f-96ec-8470267bf343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcb88983a8e1439588644b71170d7362'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:28.731799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e7c6a09e-d909-4b8f-96ec-8470267bf343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcb88983a8e1439588644b71170d7362'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:28.738754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e7c6a09e-d909-4b8f-96ec-8470267bf343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcb88983a8e1439588644b71170d7362'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:28.744872 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1313/2639] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:22:28 2022] DELETE /identity/v3/users/dcb88983a8e1439588644b71170d7362 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:28.970035 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b6044-880a-4392-b243-cf7dcd76c4c4 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:28.983394 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b6044-880a-4392-b243-cf7dcd76c4c4 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:29.058480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:29.070936 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564f8fc-4262-4ed7-b09b-83365ebd1634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:29.071613 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564f8fc-4262-4ed7-b09b-83365ebd1634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.071894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564f8fc-4262-4ed7-b09b-83365ebd1634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.072165 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564f8fc-4262-4ed7-b09b-83365ebd1634 admin admin] PATH_INFO: `/v3/projects/6b24b42ca93941aa8e70f5fa10303f4d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.073340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564f8fc-4262-4ed7-b09b-83365ebd1634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b24b42ca93941aa8e70f5fa10303f4d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:29.075274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564f8fc-4262-4ed7-b09b-83365ebd1634 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:29.129550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8564f8fc-4262-4ed7-b09b-83365ebd1634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b24b42ca93941aa8e70f5fa10303f4d', 'request_id': 'req-8564f8fc-4262-4ed7-b09b-83365ebd1634'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:29.138962 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8564f8fc-4262-4ed7-b09b-83365ebd1634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b24b42ca93941aa8e70f5fa10303f4d', 'request_id': 'req-8564f8fc-4262-4ed7-b09b-83365ebd1634'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:29.147887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8564f8fc-4262-4ed7-b09b-83365ebd1634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b24b42ca93941aa8e70f5fa10303f4d', 'request_id': 'req-8564f8fc-4262-4ed7-b09b-83365ebd1634'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:29.161269 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8564f8fc-4262-4ed7-b09b-83365ebd1634 admin admin] The token cache is being invalidate because project 6b24b42ca93941aa8e70f5fa10303f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:22:29.161647 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8564f8fc-4262-4ed7-b09b-83365ebd1634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:29.162315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8564f8fc-4262-4ed7-b09b-83365ebd1634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:29.163829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1327/2640] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:22:28 2022] DELETE /identity/v3/projects/6b24b42ca93941aa8e70f5fa10303f4d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:29.169637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02796907-f6ef-4b3c-9b6e-ef55d58c8605 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.169909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02796907-f6ef-4b3c-9b6e-ef55d58c8605 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.170187 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02796907-f6ef-4b3c-9b6e-ef55d58c8605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.190463 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-02796907-f6ef-4b3c-9b6e-ef55d58c8605 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:29.191720 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-02796907-f6ef-4b3c-9b6e-ef55d58c8605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:29.252410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1314/2641] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:29.260083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef6b8e-552e-4efa-9765-7b3833f33e6f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.260410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef6b8e-552e-4efa-9765-7b3833f33e6f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.260706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef6b8e-552e-4efa-9765-7b3833f33e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.281658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9eef6b8e-552e-4efa-9765-7b3833f33e6f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:29.283184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9eef6b8e-552e-4efa-9765-7b3833f33e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:29.291247 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1328/2642] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:29.295418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02796907-f6ef-4b3c-9b6e-ef55d58c8605 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:29.305548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02796907-f6ef-4b3c-9b6e-ef55d58c8605 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:29.310084 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80cedd-fd08-48f6-ae39-bbd611673bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:29.310685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80cedd-fd08-48f6-ae39-bbd611673bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.310959 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80cedd-fd08-48f6-ae39-bbd611673bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.311267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80cedd-fd08-48f6-ae39-bbd611673bc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.311963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80cedd-fd08-48f6-ae39-bbd611673bc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:22:29.312433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80cedd-fd08-48f6-ae39-bbd611673bc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:29.314263 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80cedd-fd08-48f6-ae39-bbd611673bc2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:29.325849 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1315/2643] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:22:29.331015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef6b8e-552e-4efa-9765-7b3833f33e6f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:29.341291 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef6b8e-552e-4efa-9765-7b3833f33e6f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:29.349804 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4ca92-c8da-43b2-b9fd-948aa445fb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:29.350667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4ca92-c8da-43b2-b9fd-948aa445fb43 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.350942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4ca92-c8da-43b2-b9fd-948aa445fb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.351244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4ca92-c8da-43b2-b9fd-948aa445fb43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.351887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4ca92-c8da-43b2-b9fd-948aa445fb43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:29.353777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4ca92-c8da-43b2-b9fd-948aa445fb43 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:29.382353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1329/2644] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:22:29 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:29.386353 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80cedd-fd08-48f6-ae39-bbd611673bc2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:29.397125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80cedd-fd08-48f6-ae39-bbd611673bc2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:29.453289 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48092c-856e-449c-8d83-a8e09f60a216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:29.453902 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48092c-856e-449c-8d83-a8e09f60a216 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.454179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48092c-856e-449c-8d83-a8e09f60a216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.454453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48092c-856e-449c-8d83-a8e09f60a216 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.455112 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48092c-856e-449c-8d83-a8e09f60a216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:29.457031 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48092c-856e-449c-8d83-a8e09f60a216 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:29.464906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:29.479506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1316/2645] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:22:29 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:29.483920 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4ca92-c8da-43b2-b9fd-948aa445fb43 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:29.494811 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4ca92-c8da-43b2-b9fd-948aa445fb43 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:29.500134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0405c1e-5407-46d2-a79f-69622a97e9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:29.500725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0405c1e-5407-46d2-a79f-69622a97e9ad admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.500998 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0405c1e-5407-46d2-a79f-69622a97e9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.501266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0405c1e-5407-46d2-a79f-69622a97e9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.501894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0405c1e-5407-46d2-a79f-69622a97e9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:29.503502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0405c1e-5407-46d2-a79f-69622a97e9ad admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:29.513271 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1330/2646] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:22:29.517807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48092c-856e-449c-8d83-a8e09f60a216 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:29.527892 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48092c-856e-449c-8d83-a8e09f60a216 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:29.532899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ed642-2c46-4f4b-bda7-5204a8b6c338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:29.533497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ed642-2c46-4f4b-bda7-5204a8b6c338 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.533780 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ed642-2c46-4f4b-bda7-5204a8b6c338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.534056 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ed642-2c46-4f4b-bda7-5204a8b6c338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.534723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ed642-2c46-4f4b-bda7-5204a8b6c338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:29.536367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ed642-2c46-4f4b-bda7-5204a8b6c338 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:29.546894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1317/2647] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:22:29.551191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0405c1e-5407-46d2-a79f-69622a97e9ad admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:29.562837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0405c1e-5407-46d2-a79f-69622a97e9ad admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:29.569463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9811d51-a977-4071-b6b6-6f508367e1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:29.570109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9811d51-a977-4071-b6b6-6f508367e1a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.570781 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9811d51-a977-4071-b6b6-6f508367e1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.571608 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9811d51-a977-4071-b6b6-6f508367e1a0 admin admin] PATH_INFO: `/v3/projects/aa567eebf0254981865a5c3732f4c3d5/users/a2999e7403fc457184cddd7340063aeb/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.586590 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9811d51-a977-4071-b6b6-6f508367e1a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa567eebf0254981865a5c3732f4c3d5, user_id=a2999e7403fc457184cddd7340063aeb, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:29.589244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9811d51-a977-4071-b6b6-6f508367e1a0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:29.603079 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1331/2648] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:29 2022] PUT /identity/v3/projects/aa567eebf0254981865a5c3732f4c3d5/users/a2999e7403fc457184cddd7340063aeb/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:29.614489 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a006a-45ed-4877-aac1-c43c3f5bc51c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.614793 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a006a-45ed-4877-aac1-c43c3f5bc51c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.615091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a006a-45ed-4877-aac1-c43c3f5bc51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.643759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4d2a006a-45ed-4877-aac1-c43c3f5bc51c None None] MFA Rules not processed for user `a2999e7403fc457184cddd7340063aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:29.644972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4d2a006a-45ed-4877-aac1-c43c3f5bc51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:29.708492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1318/2649] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:29.713019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9811d51-a977-4071-b6b6-6f508367e1a0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:29.774025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9811d51-a977-4071-b6b6-6f508367e1a0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:29.778788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ee659-9c9d-4bb3-8925-c7c63f91cef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:29.779473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ee659-9c9d-4bb3-8925-c7c63f91cef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.779755 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ee659-9c9d-4bb3-8925-c7c63f91cef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.780030 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ee659-9c9d-4bb3-8925-c7c63f91cef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.780663 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ee659-9c9d-4bb3-8925-c7c63f91cef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:29.782723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ee659-9c9d-4bb3-8925-c7c63f91cef7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:29.809177 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1332/2650] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:22:29 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:29.813857 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a006a-45ed-4877-aac1-c43c3f5bc51c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:29.824422 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a006a-45ed-4877-aac1-c43c3f5bc51c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:29.887191 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61cebb-201d-4b60-ae19-76608d3d760c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:29.887522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61cebb-201d-4b60-ae19-76608d3d760c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.888138 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61cebb-201d-4b60-ae19-76608d3d760c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.888138 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61cebb-201d-4b60-ae19-76608d3d760c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.888701 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61cebb-201d-4b60-ae19-76608d3d760c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:29.891066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61cebb-201d-4b60-ae19-76608d3d760c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:29.913238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1319/2651] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:22:29 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:29.917149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ee659-9c9d-4bb3-8925-c7c63f91cef7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:29.928473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ee659-9c9d-4bb3-8925-c7c63f91cef7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:29.933194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bba82-6318-41e8-ae13-32ce9fb8d518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:29.933836 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bba82-6318-41e8-ae13-32ce9fb8d518 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.934115 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bba82-6318-41e8-ae13-32ce9fb8d518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.934370 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bba82-6318-41e8-ae13-32ce9fb8d518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.935013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bba82-6318-41e8-ae13-32ce9fb8d518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:29.937897 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bba82-6318-41e8-ae13-32ce9fb8d518 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:29.939496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:29.952104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1333/2652] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:22:29.956951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61cebb-201d-4b60-ae19-76608d3d760c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:29.967171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61cebb-201d-4b60-ae19-76608d3d760c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:29.971918 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87121dbd-d413-4468-9c17-e0daae3f3c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:29.972566 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87121dbd-d413-4468-9c17-e0daae3f3c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:29.972848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87121dbd-d413-4468-9c17-e0daae3f3c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:29.973127 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87121dbd-d413-4468-9c17-e0daae3f3c96 admin admin] PATH_INFO: `/v3/projects/4b3ebe4244324b55821c46e3d0dc352a/users/965cd45a76004e20a7c4738619fc8373/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:29.988472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87121dbd-d413-4468-9c17-e0daae3f3c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b3ebe4244324b55821c46e3d0dc352a, user_id=965cd45a76004e20a7c4738619fc8373, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:29.991247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87121dbd-d413-4468-9c17-e0daae3f3c96 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:30.004826 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1320/2653] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:29 2022] PUT /identity/v3/projects/4b3ebe4244324b55821c46e3d0dc352a/users/965cd45a76004e20a7c4738619fc8373/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:30.008748 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bba82-6318-41e8-ae13-32ce9fb8d518 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:30.018471 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bba82-6318-41e8-ae13-32ce9fb8d518 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:30.075418 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6f835-ac58-4302-891d-13c6a4e69c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:30.076064 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6f835-ac58-4302-891d-13c6a4e69c24 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:30.076361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6f835-ac58-4302-891d-13c6a4e69c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:30.076619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6f835-ac58-4302-891d-13c6a4e69c24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:30.077257 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6f835-ac58-4302-891d-13c6a4e69c24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:30.078845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6f835-ac58-4302-891d-13c6a4e69c24 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:30.088595 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1334/2654] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:22:30.093377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87121dbd-d413-4468-9c17-e0daae3f3c96 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:30.103593 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87121dbd-d413-4468-9c17-e0daae3f3c96 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:30.108189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d0491-24c6-424d-902f-9501be17c482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:30.109359 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d0491-24c6-424d-902f-9501be17c482 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:30.109638 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d0491-24c6-424d-902f-9501be17c482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:30.110060 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d0491-24c6-424d-902f-9501be17c482 admin admin] PATH_INFO: `/v3/projects/4b3ebe4244324b55821c46e3d0dc352a/users/965cd45a76004e20a7c4738619fc8373/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:30.112013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d0491-24c6-424d-902f-9501be17c482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b3ebe4244324b55821c46e3d0dc352a, user_id=965cd45a76004e20a7c4738619fc8373, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:30.114640 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d0491-24c6-424d-902f-9501be17c482 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:30.128055 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1321/2655] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:30 2022] PUT /identity/v3/projects/4b3ebe4244324b55821c46e3d0dc352a/users/965cd45a76004e20a7c4738619fc8373/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:30.138141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b1548-91f0-47e5-8e8d-9ce7fefa547d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:30.138494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b1548-91f0-47e5-8e8d-9ce7fefa547d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:30.138835 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b1548-91f0-47e5-8e8d-9ce7fefa547d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:30.168749 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c23b1548-91f0-47e5-8e8d-9ce7fefa547d None None] MFA Rules not processed for user `965cd45a76004e20a7c4738619fc8373`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:30.170005 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c23b1548-91f0-47e5-8e8d-9ce7fefa547d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:30.244031 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1335/2656] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:30.254964 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d0491-24c6-424d-902f-9501be17c482 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:30.256840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a79d0491-24c6-424d-902f-9501be17c482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:30.281462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:30.320558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d0491-24c6-424d-902f-9501be17c482 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:30.324307 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84753c5f-2750-4133-9720-1f915f1725a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:30.325000 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84753c5f-2750-4133-9720-1f915f1725a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:30.325285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84753c5f-2750-4133-9720-1f915f1725a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:30.325546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84753c5f-2750-4133-9720-1f915f1725a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:30.334665 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84753c5f-2750-4133-9720-1f915f1725a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:30.336190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84753c5f-2750-4133-9720-1f915f1725a6 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:30.346054 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1322/2657] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:31.713750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b1548-91f0-47e5-8e8d-9ce7fefa547d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:31.786599 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b1548-91f0-47e5-8e8d-9ce7fefa547d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:31.791747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:31.791747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:31.792033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:31.792292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] PATH_INFO: `/v3/users/a2999e7403fc457184cddd7340063aeb` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:31.793415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2999e7403fc457184cddd7340063aeb)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:31.796956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:31.823495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:31.856349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2999e7403fc457184cddd7340063aeb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:31.865686 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2999e7403fc457184cddd7340063aeb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:31.866444 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2999e7403fc457184cddd7340063aeb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:31.884800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2999e7403fc457184cddd7340063aeb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:31.895875 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2999e7403fc457184cddd7340063aeb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:31.911087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1336/2658] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:22:31 2022] DELETE /identity/v3/users/a2999e7403fc457184cddd7340063aeb => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:31.912431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e282f7-b855-4b4b-8930-abb24ac42709 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:31.912888 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e282f7-b855-4b4b-8930-abb24ac42709 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:31.912971 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e282f7-b855-4b4b-8930-abb24ac42709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:31.935492 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a9e282f7-b855-4b4b-8930-abb24ac42709 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:31.938042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a9e282f7-b855-4b4b-8930-abb24ac42709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:32.012008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1323/2659] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:22:32.022041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:32.023465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:32.083881 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c3afe-f879-45af-a934-8bb76042f821 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:32.087701 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b231695-0d51-4a60-8c6e-f46cf1fa00ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:32.088313 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b231695-0d51-4a60-8c6e-f46cf1fa00ab service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:32.088591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b231695-0d51-4a60-8c6e-f46cf1fa00ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:32.088865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b231695-0d51-4a60-8c6e-f46cf1fa00ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:32.098911 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b231695-0d51-4a60-8c6e-f46cf1fa00ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:32.100386 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b231695-0d51-4a60-8c6e-f46cf1fa00ab service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:32.110987 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1337/2660] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:32.399777 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e282f7-b855-4b4b-8930-abb24ac42709 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:32.401189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a9e282f7-b855-4b4b-8930-abb24ac42709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:32.415206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e282f7-b855-4b4b-8930-abb24ac42709 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:32.419939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:32.420829 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:32.420829 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:32.421154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] PATH_INFO: `/v3/projects/aa567eebf0254981865a5c3732f4c3d5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:32.423089 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa567eebf0254981865a5c3732f4c3d5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:32.424760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:32.484169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfdae7-d37f-475d-8d34-756984b095f9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:32.484545 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfdae7-d37f-475d-8d34-756984b095f9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:32.484818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfdae7-d37f-475d-8d34-756984b095f9 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:32.486419 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1338/2661] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 13:22:32.486541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa567eebf0254981865a5c3732f4c3d5', 'request_id': 'req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:32.492757 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c5cec-608a-46f7-a469-065835716e23 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:32.493127 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c5cec-608a-46f7-a469-065835716e23 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:32.493502 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c5cec-608a-46f7-a469-065835716e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:32.493610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa567eebf0254981865a5c3732f4c3d5', 'request_id': 'req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:32.501193 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa567eebf0254981865a5c3732f4c3d5', 'request_id': 'req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:32.515827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] The token cache is being invalidate because project aa567eebf0254981865a5c3732f4c3d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:22:32.516348 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:32.517161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:32.518879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1324/2662] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:22:32 2022] DELETE /identity/v3/projects/aa567eebf0254981865a5c3732f4c3d5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:22:32.520559 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a05c5cec-608a-46f7-a469-065835716e23 None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:32.521842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a05c5cec-608a-46f7-a469-065835716e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:32.522552 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:32.523885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:32.588618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:32.607351 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24eb5e-e5ce-4f94-a1fb-e7db99a261ed admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:32.614706 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1339/2663] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:22:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:22:32.618401 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e720f-4b0e-4115-84ee-192c33bd50c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:32.619174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e720f-4b0e-4115-84ee-192c33bd50c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:32.619510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e720f-4b0e-4115-84ee-192c33bd50c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:32.619851 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e720f-4b0e-4115-84ee-192c33bd50c4 admin admin] PATH_INFO: `/v3/users/965cd45a76004e20a7c4738619fc8373` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:32.621538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e720f-4b0e-4115-84ee-192c33bd50c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=965cd45a76004e20a7c4738619fc8373)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:32.623972 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e720f-4b0e-4115-84ee-192c33bd50c4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:32.628774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c5cec-608a-46f7-a469-065835716e23 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:32.630239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a05c5cec-608a-46f7-a469-065835716e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:32.672445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:32.686392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-876e720f-4b0e-4115-84ee-192c33bd50c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '965cd45a76004e20a7c4738619fc8373'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:32.694193 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-876e720f-4b0e-4115-84ee-192c33bd50c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '965cd45a76004e20a7c4738619fc8373'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:32.698750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c5cec-608a-46f7-a469-065835716e23 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:32.702831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f156aa-b5c9-4bde-82fa-1e712e7a6f3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ef6cd64182474787134e3227beb3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:32.703654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f156aa-b5c9-4bde-82fa-1e712e7a6f3d service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:32.704032 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f156aa-b5c9-4bde-82fa-1e712e7a6f3d service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:32.704401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f156aa-b5c9-4bde-82fa-1e712e7a6f3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:32.707189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-876e720f-4b0e-4115-84ee-192c33bd50c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '965cd45a76004e20a7c4738619fc8373'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:32.707866 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-876e720f-4b0e-4115-84ee-192c33bd50c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '965cd45a76004e20a7c4738619fc8373'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:32.715026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f156aa-b5c9-4bde-82fa-1e712e7a6f3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:32.715659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-876e720f-4b0e-4115-84ee-192c33bd50c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '965cd45a76004e20a7c4738619fc8373'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:32.716488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f156aa-b5c9-4bde-82fa-1e712e7a6f3d service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:32.722086 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1325/2664] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:22:32 2022] DELETE /identity/v3/users/965cd45a76004e20a7c4738619fc8373 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:22:32.728080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1340/2665] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:32.798675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3c551-cd42-4a61-a12f-ae1b01c56375 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:32.798965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3c551-cd42-4a61-a12f-ae1b01c56375 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:32.799272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3c551-cd42-4a61-a12f-ae1b01c56375 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:32.800639 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1326/2666] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:22:32.805582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93957f2-1335-4d20-a37f-233ee25819f0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:32.805928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93957f2-1335-4d20-a37f-233ee25819f0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:32.806205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93957f2-1335-4d20-a37f-233ee25819f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:32.825640 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a93957f2-1335-4d20-a37f-233ee25819f0 None None] MFA Rules not processed for user `e12deabe077b4095bfa7e2e8fac61d14`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:32.826853 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a93957f2-1335-4d20-a37f-233ee25819f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:32.881796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1341/2667] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:22:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:32.894359 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3c551-cd42-4a61-a12f-ae1b01c56375 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:32.896198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2be3c551-cd42-4a61-a12f-ae1b01c56375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:32.910131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3c551-cd42-4a61-a12f-ae1b01c56375 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:32.913752 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70da20d-ef8f-4248-9035-1dfb346a5a89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:32.914366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70da20d-ef8f-4248-9035-1dfb346a5a89 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:32.914763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70da20d-ef8f-4248-9035-1dfb346a5a89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:32.915059 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70da20d-ef8f-4248-9035-1dfb346a5a89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:32.922646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70da20d-ef8f-4248-9035-1dfb346a5a89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:32.924133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70da20d-ef8f-4248-9035-1dfb346a5a89 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:32.932173 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1327/2668] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:33.003411 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93957f2-1335-4d20-a37f-233ee25819f0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:33.004873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a93957f2-1335-4d20-a37f-233ee25819f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:33.022034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdf737-f4a0-4bb4-a165-c1d9329c9d00 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.022394 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdf737-f4a0-4bb4-a165-c1d9329c9d00 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.022714 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdf737-f4a0-4bb4-a165-c1d9329c9d00 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.024074 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1328/2669] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 9 13:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:22:33.036199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed92799-3c65-46da-ae82-7cdc49cbceea None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.036515 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed92799-3c65-46da-ae82-7cdc49cbceea None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.036809 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed92799-3c65-46da-ae82-7cdc49cbceea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.058077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6ed92799-3c65-46da-ae82-7cdc49cbceea None None] MFA Rules not processed for user `441575b8b496457b905e9d5f5846e092`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:33.059310 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6ed92799-3c65-46da-ae82-7cdc49cbceea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:33.086691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93957f2-1335-4d20-a37f-233ee25819f0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:33.092559 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:33.093806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.094713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.095147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f admin admin] PATH_INFO: `/v3/projects/4b3ebe4244324b55821c46e3d0dc352a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.096841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b3ebe4244324b55821c46e3d0dc352a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:33.099544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:33.133462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1329/2670] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 9 13:22:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:22:33.145214 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed92799-3c65-46da-ae82-7cdc49cbceea None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:33.152619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b3ebe4244324b55821c46e3d0dc352a', 'request_id': 'req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:33.154370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed92799-3c65-46da-ae82-7cdc49cbceea None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:33.161669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b3ebe4244324b55821c46e3d0dc352a', 'request_id': 'req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:33.167680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b3ebe4244324b55821c46e3d0dc352a', 'request_id': 'req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:33.180608 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f admin admin] The token cache is being invalidate because project 4b3ebe4244324b55821c46e3d0dc352a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:22:33.181019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:33.181733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-025dcccd-55d7-41fc-bc7d-e1456ab2da0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:33.183268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1342/2671] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:22:33 2022] DELETE /identity/v3/projects/4b3ebe4244324b55821c46e3d0dc352a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:33.189141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b5edd-6133-4195-839f-156c3e3a766e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.189528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b5edd-6133-4195-839f-156c3e3a766e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.189801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b5edd-6133-4195-839f-156c3e3a766e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.203977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:33.209858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a90b5edd-6133-4195-839f-156c3e3a766e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:33.211129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a90b5edd-6133-4195-839f-156c3e3a766e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:33.214081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4279050-b0ad-4d86-bcc7-2472a07c0d84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:33.215151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4279050-b0ad-4d86-bcc7-2472a07c0d84 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.215748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4279050-b0ad-4d86-bcc7-2472a07c0d84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.216030 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4279050-b0ad-4d86-bcc7-2472a07c0d84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.217804 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f4279050-b0ad-4d86-bcc7-2472a07c0d84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:33.219852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:33.282613 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1343/2672] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:22:33.284093 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d96b-85f0-4bfc-91f7-291799bc39a8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.284372 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d96b-85f0-4bfc-91f7-291799bc39a8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.284643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d96b-85f0-4bfc-91f7-291799bc39a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.290877 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4279050-b0ad-4d86-bcc7-2472a07c0d84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:33.292355 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4279050-b0ad-4d86-bcc7-2472a07c0d84 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:33.303392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1330/2673] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:22:33.303831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d4f1d96b-85f0-4bfc-91f7-291799bc39a8 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:33.305054 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d4f1d96b-85f0-4bfc-91f7-291799bc39a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:33.313415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1344/2674] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:22:33.317567 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4279050-b0ad-4d86-bcc7-2472a07c0d84 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:33.328607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4279050-b0ad-4d86-bcc7-2472a07c0d84 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:33.333753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486b909-d203-4dde-a63e-89963ec54644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:33.334004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486b909-d203-4dde-a63e-89963ec54644 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.334283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486b909-d203-4dde-a63e-89963ec54644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.334555 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486b909-d203-4dde-a63e-89963ec54644 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.335235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486b909-d203-4dde-a63e-89963ec54644 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:22:33.336027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486b909-d203-4dde-a63e-89963ec54644 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:33.337229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486b909-d203-4dde-a63e-89963ec54644 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:33.352125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1331/2675] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:22:33.353200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1d96b-85f0-4bfc-91f7-291799bc39a8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:33.362709 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1d96b-85f0-4bfc-91f7-291799bc39a8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:33.367237 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9502d07-a26c-4554-843c-840819135c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:33.367845 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9502d07-a26c-4554-843c-840819135c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.368120 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9502d07-a26c-4554-843c-840819135c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.368388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9502d07-a26c-4554-843c-840819135c11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.369011 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9502d07-a26c-4554-843c-840819135c11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:33.370886 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9502d07-a26c-4554-843c-840819135c11 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:33.402609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1345/2676] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:22:33 2022] POST /identity/v3/projects => generated 401 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:22:33.403735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486b909-d203-4dde-a63e-89963ec54644 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:33.413689 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486b909-d203-4dde-a63e-89963ec54644 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:33.474412 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafed82-37e8-4134-acb5-3001cf3df7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:33.475034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafed82-37e8-4134-acb5-3001cf3df7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.475342 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafed82-37e8-4134-acb5-3001cf3df7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.475619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafed82-37e8-4134-acb5-3001cf3df7e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.476261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafed82-37e8-4134-acb5-3001cf3df7e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:33.478191 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafed82-37e8-4134-acb5-3001cf3df7e3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:33.503536 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1332/2677] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:22:33 2022] POST /identity/v3/users => generated 301 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:22:33.504704 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9502d07-a26c-4554-843c-840819135c11 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:33.515218 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9502d07-a26c-4554-843c-840819135c11 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:33.520605 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b26dc-287b-4787-825b-efadc2b9814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:33.521231 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b26dc-287b-4787-825b-efadc2b9814d admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.521504 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b26dc-287b-4787-825b-efadc2b9814d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.521772 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b26dc-287b-4787-825b-efadc2b9814d admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.522437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b26dc-287b-4787-825b-efadc2b9814d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:33.524169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b26dc-287b-4787-825b-efadc2b9814d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:33.538365 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1346/2678] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:22:33.539587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafed82-37e8-4134-acb5-3001cf3df7e3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:33.550343 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafed82-37e8-4134-acb5-3001cf3df7e3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:33.555848 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e905dd-6988-44b2-abd0-b94727742f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:33.556465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e905dd-6988-44b2-abd0-b94727742f77 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.556749 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e905dd-6988-44b2-abd0-b94727742f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.557026 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e905dd-6988-44b2-abd0-b94727742f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.557676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e905dd-6988-44b2-abd0-b94727742f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:33.559325 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e905dd-6988-44b2-abd0-b94727742f77 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:33.575942 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1333/2679] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:22:33.577321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b26dc-287b-4787-825b-efadc2b9814d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:33.588168 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b26dc-287b-4787-825b-efadc2b9814d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:33.592843 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924001a5-28f5-4087-8f0d-9087f41006ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:33.594179 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924001a5-28f5-4087-8f0d-9087f41006ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.594712 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924001a5-28f5-4087-8f0d-9087f41006ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.595013 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924001a5-28f5-4087-8f0d-9087f41006ab admin admin] PATH_INFO: `/v3/projects/171c7b0959c348028a58b76df0762652/users/91010dc06bdd48598b716835f57b8454/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.610484 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924001a5-28f5-4087-8f0d-9087f41006ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=171c7b0959c348028a58b76df0762652, user_id=91010dc06bdd48598b716835f57b8454, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:33.613181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924001a5-28f5-4087-8f0d-9087f41006ab admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:33.627964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1347/2680] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:33 2022] PUT /identity/v3/projects/171c7b0959c348028a58b76df0762652/users/91010dc06bdd48598b716835f57b8454/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:33.639238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2582aa-9906-49a2-a3eb-4a1411332ad5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.639520 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2582aa-9906-49a2-a3eb-4a1411332ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.639787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2582aa-9906-49a2-a3eb-4a1411332ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.672020 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fc2582aa-9906-49a2-a3eb-4a1411332ad5 None None] MFA Rules not processed for user `91010dc06bdd48598b716835f57b8454`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:33.673236 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fc2582aa-9906-49a2-a3eb-4a1411332ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:33.717391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:33.746364 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1334/2681] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:33 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:33.752061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924001a5-28f5-4087-8f0d-9087f41006ab admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:33.828628 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924001a5-28f5-4087-8f0d-9087f41006ab admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:33.844326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a3b63-8d0f-4373-99b5-eaefd885b477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:33.845110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a3b63-8d0f-4373-99b5-eaefd885b477 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.845555 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a3b63-8d0f-4373-99b5-eaefd885b477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.845927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a3b63-8d0f-4373-99b5-eaefd885b477 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.846662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a3b63-8d0f-4373-99b5-eaefd885b477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:33.848740 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a3b63-8d0f-4373-99b5-eaefd885b477 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:33.878085 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1348/2682] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:22:33 2022] POST /identity/v3/projects => generated 401 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:33.882206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2582aa-9906-49a2-a3eb-4a1411332ad5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:33.893545 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2582aa-9906-49a2-a3eb-4a1411332ad5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:33.951356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85024246-6e74-40d5-886a-c2806e73d028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:33.952222 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85024246-6e74-40d5-886a-c2806e73d028 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.952721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85024246-6e74-40d5-886a-c2806e73d028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.953007 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85024246-6e74-40d5-886a-c2806e73d028 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.953769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85024246-6e74-40d5-886a-c2806e73d028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:33.955728 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85024246-6e74-40d5-886a-c2806e73d028 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:33.977951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1335/2683] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:22:33 2022] POST /identity/v3/users => generated 307 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:33.981966 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a3b63-8d0f-4373-99b5-eaefd885b477 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:33.993061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a3b63-8d0f-4373-99b5-eaefd885b477 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:33.997850 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2f19f-bbfe-4cce-8a09-2e7083951527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:33.998322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2f19f-bbfe-4cce-8a09-2e7083951527 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:33.998855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2f19f-bbfe-4cce-8a09-2e7083951527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:33.999158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2f19f-bbfe-4cce-8a09-2e7083951527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:33.999834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2f19f-bbfe-4cce-8a09-2e7083951527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:34.001419 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2f19f-bbfe-4cce-8a09-2e7083951527 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:34.012204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1349/2684] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:22:34.016623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85024246-6e74-40d5-886a-c2806e73d028 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:34.026536 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85024246-6e74-40d5-886a-c2806e73d028 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:34.031890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c974a-47c0-47ba-941a-2dc4db4cb4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:34.032543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c974a-47c0-47ba-941a-2dc4db4cb4d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:34.032823 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c974a-47c0-47ba-941a-2dc4db4cb4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:34.033128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c974a-47c0-47ba-941a-2dc4db4cb4d5 admin admin] PATH_INFO: `/v3/projects/57f66a9594ac4445a71a3806ee5d4a28/users/042690219d3d4ac8a5f572cf9f7ac627/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:34.048340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c974a-47c0-47ba-941a-2dc4db4cb4d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f66a9594ac4445a71a3806ee5d4a28, user_id=042690219d3d4ac8a5f572cf9f7ac627, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:34.050996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c974a-47c0-47ba-941a-2dc4db4cb4d5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:34.064434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1336/2685] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:34 2022] PUT /identity/v3/projects/57f66a9594ac4445a71a3806ee5d4a28/users/042690219d3d4ac8a5f572cf9f7ac627/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:34.068600 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2f19f-bbfe-4cce-8a09-2e7083951527 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:34.079326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2f19f-bbfe-4cce-8a09-2e7083951527 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:34.100330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:34.137887 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa46e51-e8f9-4088-b6ba-325a76bbf4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:34.138528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa46e51-e8f9-4088-b6ba-325a76bbf4c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:34.138808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa46e51-e8f9-4088-b6ba-325a76bbf4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:34.139104 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa46e51-e8f9-4088-b6ba-325a76bbf4c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:34.139750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa46e51-e8f9-4088-b6ba-325a76bbf4c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:34.141916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa46e51-e8f9-4088-b6ba-325a76bbf4c9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:34.151513 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1350/2686] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:22:34.155940 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c974a-47c0-47ba-941a-2dc4db4cb4d5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:34.166863 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c974a-47c0-47ba-941a-2dc4db4cb4d5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:34.172428 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f68910-076a-4c1b-997f-87c550c183b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:34.173123 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f68910-076a-4c1b-997f-87c550c183b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:34.173397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f68910-076a-4c1b-997f-87c550c183b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:34.173666 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f68910-076a-4c1b-997f-87c550c183b6 admin admin] PATH_INFO: `/v3/projects/57f66a9594ac4445a71a3806ee5d4a28/users/042690219d3d4ac8a5f572cf9f7ac627/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:34.175627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f68910-076a-4c1b-997f-87c550c183b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f66a9594ac4445a71a3806ee5d4a28, user_id=042690219d3d4ac8a5f572cf9f7ac627, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:34.178244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f68910-076a-4c1b-997f-87c550c183b6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:34.189407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1337/2687] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:34 2022] PUT /identity/v3/projects/57f66a9594ac4445a71a3806ee5d4a28/users/042690219d3d4ac8a5f572cf9f7ac627/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:34.199716 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ada3d7-15ac-4df5-bc04-cc63778b0ac0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:34.200005 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ada3d7-15ac-4df5-bc04-cc63778b0ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:34.200286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ada3d7-15ac-4df5-bc04-cc63778b0ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:34.229820 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a2ada3d7-15ac-4df5-bc04-cc63778b0ac0 None None] MFA Rules not processed for user `042690219d3d4ac8a5f572cf9f7ac627`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:34.231103 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a2ada3d7-15ac-4df5-bc04-cc63778b0ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:34.303643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1351/2688] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:34 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:34.315217 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f68910-076a-4c1b-997f-87c550c183b6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:34.317015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b3f68910-076a-4c1b-997f-87c550c183b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:34.376489 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f68910-076a-4c1b-997f-87c550c183b6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:34.380055 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67311e91-b18a-4877-9bb5-4548a24d947f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:34.380670 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67311e91-b18a-4877-9bb5-4548a24d947f service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:34.380945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67311e91-b18a-4877-9bb5-4548a24d947f service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:34.381213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67311e91-b18a-4877-9bb5-4548a24d947f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:34.390329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67311e91-b18a-4877-9bb5-4548a24d947f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:34.391793 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67311e91-b18a-4877-9bb5-4548a24d947f service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:34.401469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1338/2689] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:34 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:34.478950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ada3d7-15ac-4df5-bc04-cc63778b0ac0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:34.489216 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ada3d7-15ac-4df5-bc04-cc63778b0ac0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:34.493191 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420449db-dd27-47d1-b8fb-5e5fe0d3356f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:34.493798 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420449db-dd27-47d1-b8fb-5e5fe0d3356f service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:34.494072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420449db-dd27-47d1-b8fb-5e5fe0d3356f service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:34.494341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420449db-dd27-47d1-b8fb-5e5fe0d3356f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:34.555248 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420449db-dd27-47d1-b8fb-5e5fe0d3356f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:34.556862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420449db-dd27-47d1-b8fb-5e5fe0d3356f service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:34.566602 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1352/2690] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:34 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:35.009393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bac996-49db-4a76-9416-a73b7c016dfd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:35.009894 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bac996-49db-4a76-9416-a73b7c016dfd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:35.009994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bac996-49db-4a76-9416-a73b7c016dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:35.029611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-18bac996-49db-4a76-9416-a73b7c016dfd None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:35.031706 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-18bac996-49db-4a76-9416-a73b7c016dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:35.072748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:35.100613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1339/2691] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:22:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:35.113117 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420449db-dd27-47d1-b8fb-5e5fe0d3356f service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:35.115179 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-420449db-dd27-47d1-b8fb-5e5fe0d3356f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:35.163826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:35.187822 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420449db-dd27-47d1-b8fb-5e5fe0d3356f service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:35.193416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d961dd-2afa-4d8d-a53c-e56c05862ae5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:35.194042 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d961dd-2afa-4d8d-a53c-e56c05862ae5 service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:35.194303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d961dd-2afa-4d8d-a53c-e56c05862ae5 service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:35.194573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d961dd-2afa-4d8d-a53c-e56c05862ae5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:35.202512 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d961dd-2afa-4d8d-a53c-e56c05862ae5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:35.204469 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d961dd-2afa-4d8d-a53c-e56c05862ae5 service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:35.212342 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1353/2692] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:22:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:22:38.803759 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bac996-49db-4a76-9416-a73b7c016dfd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:38.872275 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bac996-49db-4a76-9416-a73b7c016dfd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:38.878116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:38.878898 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:38.879220 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:38.879500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] PATH_INFO: `/v3/users/91010dc06bdd48598b716835f57b8454` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:38.880654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91010dc06bdd48598b716835f57b8454)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:38.882796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:38.940007 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91010dc06bdd48598b716835f57b8454'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:38.949721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91010dc06bdd48598b716835f57b8454'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:38.964878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91010dc06bdd48598b716835f57b8454'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:38.965609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91010dc06bdd48598b716835f57b8454'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:38.986202 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91010dc06bdd48598b716835f57b8454'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:38.992977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1340/2693] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:22:38 2022] DELETE /identity/v3/users/91010dc06bdd48598b716835f57b8454 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:38.997507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63ddf6-1072-447d-81c7-a16a19212dc7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:38.998066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63ddf6-1072-447d-81c7-a16a19212dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:38.998400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63ddf6-1072-447d-81c7-a16a19212dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:39.020258 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-aa63ddf6-1072-447d-81c7-a16a19212dc7 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:39.021589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aa63ddf6-1072-447d-81c7-a16a19212dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:39.086124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1354/2694] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:39.096602 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:39.098038 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:39.163029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45fb50-5c03-47d6-b4ff-a4df8f039654 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:39.166988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdbf4f-5740-478e-97f2-f2103cf6e36c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:39.167660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdbf4f-5740-478e-97f2-f2103cf6e36c service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:39.167943 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdbf4f-5740-478e-97f2-f2103cf6e36c service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:39.168219 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdbf4f-5740-478e-97f2-f2103cf6e36c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:39.178328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdbf4f-5740-478e-97f2-f2103cf6e36c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:39.179847 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdbf4f-5740-478e-97f2-f2103cf6e36c service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:39.190746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1341/2695] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:39.489145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63ddf6-1072-447d-81c7-a16a19212dc7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:39.490690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aa63ddf6-1072-447d-81c7-a16a19212dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:39.505790 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63ddf6-1072-447d-81c7-a16a19212dc7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:39.512317 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:39.513458 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:39.514072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:39.514564 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] PATH_INFO: `/v3/projects/171c7b0959c348028a58b76df0762652` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:39.516762 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=171c7b0959c348028a58b76df0762652)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:39.518619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:39.601508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '171c7b0959c348028a58b76df0762652', 'request_id': 'req-f43fde31-65fe-45ed-b20b-5458f13033a4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:39.612162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '171c7b0959c348028a58b76df0762652', 'request_id': 'req-f43fde31-65fe-45ed-b20b-5458f13033a4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:39.620988 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '171c7b0959c348028a58b76df0762652', 'request_id': 'req-f43fde31-65fe-45ed-b20b-5458f13033a4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:39.637145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] The token cache is being invalidate because project 171c7b0959c348028a58b76df0762652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:22:39.637533 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:39.638383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:39.639830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1355/2696] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:22:39 2022] DELETE /identity/v3/projects/171c7b0959c348028a58b76df0762652 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:39.644341 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdbf4f-5740-478e-97f2-f2103cf6e36c service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:39.649701 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:39.669812 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdbf4f-5740-478e-97f2-f2103cf6e36c service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:39.765879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119b305-69fd-4ad2-9b0c-d32d7fab4f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:39.768143 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119b305-69fd-4ad2-9b0c-d32d7fab4f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:39.768475 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119b305-69fd-4ad2-9b0c-d32d7fab4f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:39.768904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119b305-69fd-4ad2-9b0c-d32d7fab4f55 admin admin] PATH_INFO: `/v3/users/042690219d3d4ac8a5f572cf9f7ac627` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:39.770340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119b305-69fd-4ad2-9b0c-d32d7fab4f55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=042690219d3d4ac8a5f572cf9f7ac627)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:39.773305 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119b305-69fd-4ad2-9b0c-d32d7fab4f55 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:39.862183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b119b305-69fd-4ad2-9b0c-d32d7fab4f55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '042690219d3d4ac8a5f572cf9f7ac627'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:39.874359 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b119b305-69fd-4ad2-9b0c-d32d7fab4f55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '042690219d3d4ac8a5f572cf9f7ac627'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:39.899355 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b119b305-69fd-4ad2-9b0c-d32d7fab4f55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '042690219d3d4ac8a5f572cf9f7ac627'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:39.900305 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b119b305-69fd-4ad2-9b0c-d32d7fab4f55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '042690219d3d4ac8a5f572cf9f7ac627'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:39.915837 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b119b305-69fd-4ad2-9b0c-d32d7fab4f55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '042690219d3d4ac8a5f572cf9f7ac627'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:39.930721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1342/2697] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:22:39 2022] DELETE /identity/v3/users/042690219d3d4ac8a5f572cf9f7ac627 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:40.276641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:40.278885 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:40.300725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fde31-65fe-45ed-b20b-5458f13033a4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:40.396565 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:40.408496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:40.409638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:40.410137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:40.411171 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025 admin admin] PATH_INFO: `/v3/projects/57f66a9594ac4445a71a3806ee5d4a28` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:40.414457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f66a9594ac4445a71a3806ee5d4a28)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:40.422094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:40.495984 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f66a9594ac4445a71a3806ee5d4a28', 'request_id': 'req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:40.511723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f66a9594ac4445a71a3806ee5d4a28', 'request_id': 'req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:40.519393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f66a9594ac4445a71a3806ee5d4a28', 'request_id': 'req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:40.546950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025 admin admin] The token cache is being invalidate because project 57f66a9594ac4445a71a3806ee5d4a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:22:40.548331 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:40.550852 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-653f6cf6-4ba9-4a16-a7d4-7ba14be2a025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:40.558548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1356/2698] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:22:40 2022] DELETE /identity/v3/projects/57f66a9594ac4445a71a3806ee5d4a28 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:40.564382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892b7e8-3f14-491e-889f-33545d991364 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:40.565224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892b7e8-3f14-491e-889f-33545d991364 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:40.566359 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892b7e8-3f14-491e-889f-33545d991364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:40.591122 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b892b7e8-3f14-491e-889f-33545d991364 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:40.593889 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b892b7e8-3f14-491e-889f-33545d991364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:40.667800 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1343/2699] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:40.677772 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a415d96-5ee0-4e92-ab4a-6bf0d343d99f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:40.677772 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a415d96-5ee0-4e92-ab4a-6bf0d343d99f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:40.678055 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a415d96-5ee0-4e92-ab4a-6bf0d343d99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:40.770320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1a415d96-5ee0-4e92-ab4a-6bf0d343d99f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:40.771574 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1a415d96-5ee0-4e92-ab4a-6bf0d343d99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:40.825713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1357/2700] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:40.830834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892b7e8-3f14-491e-889f-33545d991364 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:40.858453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:40.889354 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892b7e8-3f14-491e-889f-33545d991364 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:40.894516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637095a7-ff9a-4342-a1b2-b361a4078ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:40.895158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637095a7-ff9a-4342-a1b2-b361a4078ecf admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:40.895484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637095a7-ff9a-4342-a1b2-b361a4078ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:40.895849 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637095a7-ff9a-4342-a1b2-b361a4078ecf admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:40.896506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637095a7-ff9a-4342-a1b2-b361a4078ecf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:22:40.896811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637095a7-ff9a-4342-a1b2-b361a4078ecf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:40.898418 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637095a7-ff9a-4342-a1b2-b361a4078ecf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:40.912561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1344/2701] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:22:40.917149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a415d96-5ee0-4e92-ab4a-6bf0d343d99f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:40.928059 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a415d96-5ee0-4e92-ab4a-6bf0d343d99f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:40.932854 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89adfe-da6f-4c53-9587-ed3aa3e4094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:40.933557 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89adfe-da6f-4c53-9587-ed3aa3e4094d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:40.933918 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89adfe-da6f-4c53-9587-ed3aa3e4094d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:40.934256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89adfe-da6f-4c53-9587-ed3aa3e4094d admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:40.934956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89adfe-da6f-4c53-9587-ed3aa3e4094d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:40.937016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89adfe-da6f-4c53-9587-ed3aa3e4094d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:40.969849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1358/2702] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:22:40 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:40.971992 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637095a7-ff9a-4342-a1b2-b361a4078ecf admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:40.982904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637095a7-ff9a-4342-a1b2-b361a4078ecf admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:41.059878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07dc0e4-698d-4c6f-bae0-7690d407087c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:41.060606 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07dc0e4-698d-4c6f-bae0-7690d407087c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:41.061042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07dc0e4-698d-4c6f-bae0-7690d407087c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:41.061374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07dc0e4-698d-4c6f-bae0-7690d407087c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:41.063295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07dc0e4-698d-4c6f-bae0-7690d407087c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:41.065376 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07dc0e4-698d-4c6f-bae0-7690d407087c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:41.101169 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1345/2703] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:22:40 2022] POST /identity/v3/users => generated 283 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:22:41.105406 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89adfe-da6f-4c53-9587-ed3aa3e4094d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:41.120692 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89adfe-da6f-4c53-9587-ed3aa3e4094d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:41.133829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3be6e-d566-43a3-bbb0-0d83598a8a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:41.134569 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3be6e-d566-43a3-bbb0-0d83598a8a06 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:41.134949 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3be6e-d566-43a3-bbb0-0d83598a8a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:41.135353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3be6e-d566-43a3-bbb0-0d83598a8a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:41.136190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3be6e-d566-43a3-bbb0-0d83598a8a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:41.138108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3be6e-d566-43a3-bbb0-0d83598a8a06 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:41.172400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1359/2704] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:22:41.173472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07dc0e4-698d-4c6f-bae0-7690d407087c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:41.186424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07dc0e4-698d-4c6f-bae0-7690d407087c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:41.193696 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a790e-0fd7-4c39-bc8e-d8345d9b8a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:41.194732 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a790e-0fd7-4c39-bc8e-d8345d9b8a7f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:41.195229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a790e-0fd7-4c39-bc8e-d8345d9b8a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:41.195611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a790e-0fd7-4c39-bc8e-d8345d9b8a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:41.196520 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a790e-0fd7-4c39-bc8e-d8345d9b8a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:41.198634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a790e-0fd7-4c39-bc8e-d8345d9b8a7f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:41.218091 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1346/2705] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:22:41.221265 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3be6e-d566-43a3-bbb0-0d83598a8a06 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:41.232850 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3be6e-d566-43a3-bbb0-0d83598a8a06 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:41.240115 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8eb2e-7c48-469c-863f-a48ad1eec502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:41.240784 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8eb2e-7c48-469c-863f-a48ad1eec502 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:41.241067 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8eb2e-7c48-469c-863f-a48ad1eec502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:41.241336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8eb2e-7c48-469c-863f-a48ad1eec502 admin admin] PATH_INFO: `/v3/projects/26e0c7bba2d247e687537afd307b6692/users/f3737b6f4d5543d59219bbcc0257a43e/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:41.254924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:41.258361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8eb2e-7c48-469c-863f-a48ad1eec502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e0c7bba2d247e687537afd307b6692, user_id=f3737b6f4d5543d59219bbcc0257a43e, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:41.261903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8eb2e-7c48-469c-863f-a48ad1eec502 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:41.277629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1360/2706] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:41 2022] PUT /identity/v3/projects/26e0c7bba2d247e687537afd307b6692/users/f3737b6f4d5543d59219bbcc0257a43e/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:41.288346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57fc2e-d627-4720-ba2c-510efec32f1d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:41.288878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57fc2e-d627-4720-ba2c-510efec32f1d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:41.289479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57fc2e-d627-4720-ba2c-510efec32f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:41.323845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0e57fc2e-d627-4720-ba2c-510efec32f1d None None] MFA Rules not processed for user `f3737b6f4d5543d59219bbcc0257a43e`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:41.325376 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0e57fc2e-d627-4720-ba2c-510efec32f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:41.398546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1347/2707] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:41.402616 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8eb2e-7c48-469c-863f-a48ad1eec502 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:41.482870 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8eb2e-7c48-469c-863f-a48ad1eec502 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:41.489541 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ad0d8-0b5e-4733-b4f0-58b7aa8baf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:41.489541 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ad0d8-0b5e-4733-b4f0-58b7aa8baf16 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:41.489541 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ad0d8-0b5e-4733-b4f0-58b7aa8baf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:41.489541 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ad0d8-0b5e-4733-b4f0-58b7aa8baf16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:41.490110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ad0d8-0b5e-4733-b4f0-58b7aa8baf16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:41.492051 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ad0d8-0b5e-4733-b4f0-58b7aa8baf16 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:41.530515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1361/2708] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:22:41 2022] POST /identity/v3/projects => generated 367 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:41.532033 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57fc2e-d627-4720-ba2c-510efec32f1d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:41.543493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57fc2e-d627-4720-ba2c-510efec32f1d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:41.580923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:41.610611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de484198-d935-4119-8f29-402df18cb068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:41.611582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de484198-d935-4119-8f29-402df18cb068 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:41.612478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de484198-d935-4119-8f29-402df18cb068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:41.612765 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de484198-d935-4119-8f29-402df18cb068 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:41.613435 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de484198-d935-4119-8f29-402df18cb068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:41.615539 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de484198-d935-4119-8f29-402df18cb068 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:41.658291 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1348/2709] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:22:41 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:41.660792 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ad0d8-0b5e-4733-b4f0-58b7aa8baf16 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:41.676691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ad0d8-0b5e-4733-b4f0-58b7aa8baf16 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:41.681857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef6279-8106-444f-a79b-452fccfb40ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:41.682480 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef6279-8106-444f-a79b-452fccfb40ac admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:41.682750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef6279-8106-444f-a79b-452fccfb40ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:41.683019 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef6279-8106-444f-a79b-452fccfb40ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:41.683766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef6279-8106-444f-a79b-452fccfb40ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:41.685318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef6279-8106-444f-a79b-452fccfb40ac admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:41.698107 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1362/2710] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:22:41.702491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de484198-d935-4119-8f29-402df18cb068 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:41.713783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de484198-d935-4119-8f29-402df18cb068 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:41.719385 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a819dd8-941a-4208-b274-6ae9e84af954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:41.720077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a819dd8-941a-4208-b274-6ae9e84af954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:41.720356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a819dd8-941a-4208-b274-6ae9e84af954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:41.720662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a819dd8-941a-4208-b274-6ae9e84af954 admin admin] PATH_INFO: `/v3/projects/ca237ae1fbe441b6893075be757083ae/users/d0bde10c7f0544acb7384ed2a9d4a463/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:41.736633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a819dd8-941a-4208-b274-6ae9e84af954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca237ae1fbe441b6893075be757083ae, user_id=d0bde10c7f0544acb7384ed2a9d4a463, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:41.739527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a819dd8-941a-4208-b274-6ae9e84af954 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:41.754724 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1349/2711] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:41 2022] PUT /identity/v3/projects/ca237ae1fbe441b6893075be757083ae/users/d0bde10c7f0544acb7384ed2a9d4a463/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:41.758635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef6279-8106-444f-a79b-452fccfb40ac admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:41.769859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef6279-8106-444f-a79b-452fccfb40ac admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:41.839476 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a9378-85d8-4655-8796-e9021ced7b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:41.840143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a9378-85d8-4655-8796-e9021ced7b70 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:41.840456 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a9378-85d8-4655-8796-e9021ced7b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:41.840736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a9378-85d8-4655-8796-e9021ced7b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:41.841520 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a9378-85d8-4655-8796-e9021ced7b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:41.843217 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a9378-85d8-4655-8796-e9021ced7b70 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:41.854847 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1363/2712] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:22:41.858460 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a819dd8-941a-4208-b274-6ae9e84af954 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:41.870638 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a819dd8-941a-4208-b274-6ae9e84af954 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:41.876729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556f6ab-c9d8-4d87-8228-031076e7e5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:41.877405 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556f6ab-c9d8-4d87-8228-031076e7e5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:41.877682 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556f6ab-c9d8-4d87-8228-031076e7e5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:41.877959 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556f6ab-c9d8-4d87-8228-031076e7e5bc admin admin] PATH_INFO: `/v3/projects/ca237ae1fbe441b6893075be757083ae/users/d0bde10c7f0544acb7384ed2a9d4a463/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:41.879945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556f6ab-c9d8-4d87-8228-031076e7e5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca237ae1fbe441b6893075be757083ae, user_id=d0bde10c7f0544acb7384ed2a9d4a463, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:41.882603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556f6ab-c9d8-4d87-8228-031076e7e5bc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:41.894407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1350/2713] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:41 2022] PUT /identity/v3/projects/ca237ae1fbe441b6893075be757083ae/users/d0bde10c7f0544acb7384ed2a9d4a463/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:22:41.904729 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8672d0-3bee-4dfb-80fe-bab40acf6e01 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:41.905012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8672d0-3bee-4dfb-80fe-bab40acf6e01 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:41.905290 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8672d0-3bee-4dfb-80fe-bab40acf6e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:41.947879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9c8672d0-3bee-4dfb-80fe-bab40acf6e01 None None] MFA Rules not processed for user `d0bde10c7f0544acb7384ed2a9d4a463`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:41.949118 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9c8672d0-3bee-4dfb-80fe-bab40acf6e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:42.002728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:42.027099 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1364/2714] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:42.038661 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556f6ab-c9d8-4d87-8228-031076e7e5bc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:42.040466 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5556f6ab-c9d8-4d87-8228-031076e7e5bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:42.126472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556f6ab-c9d8-4d87-8228-031076e7e5bc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:42.130561 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd44a4-b330-46a2-89b4-f40757823a80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:42.131278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd44a4-b330-46a2-89b4-f40757823a80 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:42.131557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd44a4-b330-46a2-89b4-f40757823a80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:42.131829 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd44a4-b330-46a2-89b4-f40757823a80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:42.141939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd44a4-b330-46a2-89b4-f40757823a80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:42.143431 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd44a4-b330-46a2-89b4-f40757823a80 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:42.153466 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1351/2715] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:42.242451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8672d0-3bee-4dfb-80fe-bab40acf6e01 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:42.264708 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8672d0-3bee-4dfb-80fe-bab40acf6e01 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:42.269933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abb86e-219d-4f4e-ae95-bbe6be0cf86d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:42.270770 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abb86e-219d-4f4e-ae95-bbe6be0cf86d service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:42.271292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abb86e-219d-4f4e-ae95-bbe6be0cf86d service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:42.272937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abb86e-219d-4f4e-ae95-bbe6be0cf86d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:42.351036 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abb86e-219d-4f4e-ae95-bbe6be0cf86d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:42.352776 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abb86e-219d-4f4e-ae95-bbe6be0cf86d service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:42.363522 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1365/2716] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:47.206607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd44a4-b330-46a2-89b4-f40757823a80 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:47.267991 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd44a4-b330-46a2-89b4-f40757823a80 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:47.272419 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:47.273037 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:47.273308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:47.273601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] PATH_INFO: `/v3/users/f3737b6f4d5543d59219bbcc0257a43e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:47.274698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3737b6f4d5543d59219bbcc0257a43e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:47.276610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:47.297587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:47.323251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3737b6f4d5543d59219bbcc0257a43e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:47.331243 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3737b6f4d5543d59219bbcc0257a43e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:47.343966 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3737b6f4d5543d59219bbcc0257a43e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:47.344590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3737b6f4d5543d59219bbcc0257a43e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:47.351643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3737b6f4d5543d59219bbcc0257a43e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:47.357434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1352/2717] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:22:47 2022] DELETE /identity/v3/users/f3737b6f4d5543d59219bbcc0257a43e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:47.362199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b366c-3e95-4eeb-b76a-ee0add04952e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:47.362688 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b366c-3e95-4eeb-b76a-ee0add04952e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:47.362802 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b366c-3e95-4eeb-b76a-ee0add04952e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:47.381603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-915b366c-3e95-4eeb-b76a-ee0add04952e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:47.382788 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-915b366c-3e95-4eeb-b76a-ee0add04952e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:47.441069 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1366/2718] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:47.450675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:47.452204 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:47.508946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa43909-3a94-423b-8660-d4b043846078 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:47.512627 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0989d9a-b9c8-4242-b34a-0a679f556a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:47.513225 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0989d9a-b9c8-4242-b34a-0a679f556a15 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:47.513503 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0989d9a-b9c8-4242-b34a-0a679f556a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:47.513774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0989d9a-b9c8-4242-b34a-0a679f556a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:47.523392 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0989d9a-b9c8-4242-b34a-0a679f556a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:47.524775 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0989d9a-b9c8-4242-b34a-0a679f556a15 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:47.534433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1353/2719] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:47.810893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b366c-3e95-4eeb-b76a-ee0add04952e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:47.812596 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-915b366c-3e95-4eeb-b76a-ee0add04952e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:47.832193 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b366c-3e95-4eeb-b76a-ee0add04952e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:47.837201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:47.837787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:47.838066 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:47.838340 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] PATH_INFO: `/v3/projects/26e0c7bba2d247e687537afd307b6692` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:47.839482 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26e0c7bba2d247e687537afd307b6692)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:47.841388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:47.889373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26e0c7bba2d247e687537afd307b6692', 'request_id': 'req-6dea474c-0739-4037-be3d-ada45a2a56d5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:47.897547 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26e0c7bba2d247e687537afd307b6692', 'request_id': 'req-6dea474c-0739-4037-be3d-ada45a2a56d5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:47.903497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26e0c7bba2d247e687537afd307b6692', 'request_id': 'req-6dea474c-0739-4037-be3d-ada45a2a56d5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:47.915190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] The token cache is being invalidate because project 26e0c7bba2d247e687537afd307b6692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:22:47.915479 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:47.916135 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:47.917627 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1367/2720] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:22:47 2022] DELETE /identity/v3/projects/26e0c7bba2d247e687537afd307b6692 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:47.922231 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0989d9a-b9c8-4242-b34a-0a679f556a15 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:47.931766 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0989d9a-b9c8-4242-b34a-0a679f556a15 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:48.006071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08374741-9896-4beb-a457-9d5442c1f59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:48.007531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08374741-9896-4beb-a457-9d5442c1f59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:48.007706 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08374741-9896-4beb-a457-9d5442c1f59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:48.009140 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08374741-9896-4beb-a457-9d5442c1f59b admin admin] PATH_INFO: `/v3/users/d0bde10c7f0544acb7384ed2a9d4a463` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:48.010699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08374741-9896-4beb-a457-9d5442c1f59b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0bde10c7f0544acb7384ed2a9d4a463)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:48.011911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08374741-9896-4beb-a457-9d5442c1f59b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:48.126967 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:48.134585 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-08374741-9896-4beb-a457-9d5442c1f59b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0bde10c7f0544acb7384ed2a9d4a463'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:48.144543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-08374741-9896-4beb-a457-9d5442c1f59b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0bde10c7f0544acb7384ed2a9d4a463'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:48.159565 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-08374741-9896-4beb-a457-9d5442c1f59b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0bde10c7f0544acb7384ed2a9d4a463'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:48.160382 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-08374741-9896-4beb-a457-9d5442c1f59b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0bde10c7f0544acb7384ed2a9d4a463'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:48.170024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-08374741-9896-4beb-a457-9d5442c1f59b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0bde10c7f0544acb7384ed2a9d4a463'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:48.176896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1354/2721] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:22:47 2022] DELETE /identity/v3/users/d0bde10c7f0544acb7384ed2a9d4a463 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:48.494223 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:48.495811 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:48.510190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea474c-0739-4037-be3d-ada45a2a56d5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:48.531027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:48.585417 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94605c6-7c00-494c-ac6a-55e5694f841c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:48.586053 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94605c6-7c00-494c-ac6a-55e5694f841c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:48.586348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94605c6-7c00-494c-ac6a-55e5694f841c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:48.586609 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94605c6-7c00-494c-ac6a-55e5694f841c admin admin] PATH_INFO: `/v3/projects/ca237ae1fbe441b6893075be757083ae` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:48.587703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94605c6-7c00-494c-ac6a-55e5694f841c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca237ae1fbe441b6893075be757083ae)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:48.589618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94605c6-7c00-494c-ac6a-55e5694f841c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:48.636474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c94605c6-7c00-494c-ac6a-55e5694f841c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca237ae1fbe441b6893075be757083ae', 'request_id': 'req-c94605c6-7c00-494c-ac6a-55e5694f841c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:48.643809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c94605c6-7c00-494c-ac6a-55e5694f841c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca237ae1fbe441b6893075be757083ae', 'request_id': 'req-c94605c6-7c00-494c-ac6a-55e5694f841c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:48.649449 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c94605c6-7c00-494c-ac6a-55e5694f841c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca237ae1fbe441b6893075be757083ae', 'request_id': 'req-c94605c6-7c00-494c-ac6a-55e5694f841c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:48.660598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c94605c6-7c00-494c-ac6a-55e5694f841c admin admin] The token cache is being invalidate because project ca237ae1fbe441b6893075be757083ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:22:48.660884 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c94605c6-7c00-494c-ac6a-55e5694f841c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:48.661522 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c94605c6-7c00-494c-ac6a-55e5694f841c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:22:48.662996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1368/2722] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:22:48 2022] DELETE /identity/v3/projects/ca237ae1fbe441b6893075be757083ae => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:48.668053 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e58eb1-bd10-4ae1-85af-9c93a983c2c8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:48.668534 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e58eb1-bd10-4ae1-85af-9c93a983c2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:48.668642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e58eb1-bd10-4ae1-85af-9c93a983c2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:48.689023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a6e58eb1-bd10-4ae1-85af-9c93a983c2c8 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:48.690199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a6e58eb1-bd10-4ae1-85af-9c93a983c2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:48.747159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1355/2723] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:22:48.755866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7d494-9c84-44b8-a3cc-811eff51ed9a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:48.756151 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7d494-9c84-44b8-a3cc-811eff51ed9a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:48.756446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7d494-9c84-44b8-a3cc-811eff51ed9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:48.775723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1ef7d494-9c84-44b8-a3cc-811eff51ed9a None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:48.776919 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1ef7d494-9c84-44b8-a3cc-811eff51ed9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:48.784713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1369/2724] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:48.788925 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e58eb1-bd10-4ae1-85af-9c93a983c2c8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:48.799079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e58eb1-bd10-4ae1-85af-9c93a983c2c8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:48.803709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad682da-986e-4b49-b939-e38eac7486b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:48.804322 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad682da-986e-4b49-b939-e38eac7486b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:48.804611 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad682da-986e-4b49-b939-e38eac7486b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:48.804881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad682da-986e-4b49-b939-e38eac7486b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:48.805532 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad682da-986e-4b49-b939-e38eac7486b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:22:48.805838 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad682da-986e-4b49-b939-e38eac7486b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:48.807497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad682da-986e-4b49-b939-e38eac7486b6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:48.819727 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1356/2725] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:22:48.823932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7d494-9c84-44b8-a3cc-811eff51ed9a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:48.833187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7d494-9c84-44b8-a3cc-811eff51ed9a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:48.837878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd8f9e-66ea-47e2-9c6e-a7dbc3df3e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:48.838474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd8f9e-66ea-47e2-9c6e-a7dbc3df3e7b admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:48.838754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd8f9e-66ea-47e2-9c6e-a7dbc3df3e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:48.839031 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd8f9e-66ea-47e2-9c6e-a7dbc3df3e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:48.839683 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd8f9e-66ea-47e2-9c6e-a7dbc3df3e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:48.841576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd8f9e-66ea-47e2-9c6e-a7dbc3df3e7b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:48.869763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1370/2726] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:22:48 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:22:48.874154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad682da-986e-4b49-b939-e38eac7486b6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:48.884541 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad682da-986e-4b49-b939-e38eac7486b6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:48.939929 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efd2e3-8319-4324-8718-1faa9a0b199f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:48.940528 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efd2e3-8319-4324-8718-1faa9a0b199f admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:48.940808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efd2e3-8319-4324-8718-1faa9a0b199f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:48.941079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efd2e3-8319-4324-8718-1faa9a0b199f admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:48.941706 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efd2e3-8319-4324-8718-1faa9a0b199f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:48.943671 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efd2e3-8319-4324-8718-1faa9a0b199f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:48.964366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1357/2727] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:22:48 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:22:48.969277 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd8f9e-66ea-47e2-9c6e-a7dbc3df3e7b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:48.979384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd8f9e-66ea-47e2-9c6e-a7dbc3df3e7b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:48.984343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ace779-c9c1-4df6-bea2-b3639f424909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:48.984931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ace779-c9c1-4df6-bea2-b3639f424909 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:48.985207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ace779-c9c1-4df6-bea2-b3639f424909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:48.985475 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ace779-c9c1-4df6-bea2-b3639f424909 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:48.986111 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ace779-c9c1-4df6-bea2-b3639f424909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:48.987724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ace779-c9c1-4df6-bea2-b3639f424909 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:48.997428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1371/2728] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:22:49.002250 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efd2e3-8319-4324-8718-1faa9a0b199f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:49.012066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efd2e3-8319-4324-8718-1faa9a0b199f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:49.016818 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a48253-e24a-4186-92b0-763eb95e0e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:49.017411 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a48253-e24a-4186-92b0-763eb95e0e19 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:49.017692 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a48253-e24a-4186-92b0-763eb95e0e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:49.017965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a48253-e24a-4186-92b0-763eb95e0e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:49.018597 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a48253-e24a-4186-92b0-763eb95e0e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:49.020253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a48253-e24a-4186-92b0-763eb95e0e19 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:49.031187 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1358/2729] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:22:49.035128 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ace779-c9c1-4df6-bea2-b3639f424909 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:49.044799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ace779-c9c1-4df6-bea2-b3639f424909 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:49.050293 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828d662-1c34-4906-94fa-71f7f7666b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:49.050928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828d662-1c34-4906-94fa-71f7f7666b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:49.051256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828d662-1c34-4906-94fa-71f7f7666b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:49.051508 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828d662-1c34-4906-94fa-71f7f7666b49 admin admin] PATH_INFO: `/v3/projects/6be36ce8b9db4257bfdfb671a298ae16/users/d90d7fe9cfc94440a603e6f010e10eb5/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:49.065761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828d662-1c34-4906-94fa-71f7f7666b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be36ce8b9db4257bfdfb671a298ae16, user_id=d90d7fe9cfc94440a603e6f010e10eb5, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:49.068421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828d662-1c34-4906-94fa-71f7f7666b49 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:49.081538 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1372/2730] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:22:49 2022] PUT /identity/v3/projects/6be36ce8b9db4257bfdfb671a298ae16/users/d90d7fe9cfc94440a603e6f010e10eb5/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:22:49.091890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6056d-d0b4-4147-97f4-24a06d9d135a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:49.092180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6056d-d0b4-4147-97f4-24a06d9d135a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:49.092533 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6056d-d0b4-4147-97f4-24a06d9d135a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:49.121175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2fb6056d-d0b4-4147-97f4-24a06d9d135a None None] MFA Rules not processed for user `d90d7fe9cfc94440a603e6f010e10eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:49.122317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2fb6056d-d0b4-4147-97f4-24a06d9d135a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:49.134384 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:49.187641 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1359/2731] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:22:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:49.198649 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828d662-1c34-4906-94fa-71f7f7666b49 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:49.200367 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f828d662-1c34-4906-94fa-71f7f7666b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:49.241457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:22:49.261208 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828d662-1c34-4906-94fa-71f7f7666b49 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:49.264806 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cf3b5-4c26-4c32-86eb-08111c108dda service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ef6cd64182474787134e3227beb3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:49.265677 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cf3b5-4c26-4c32-86eb-08111c108dda service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:49.265951 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cf3b5-4c26-4c32-86eb-08111c108dda service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:49.266220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cf3b5-4c26-4c32-86eb-08111c108dda service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:49.274371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cf3b5-4c26-4c32-86eb-08111c108dda service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:49.275818 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cf3b5-4c26-4c32-86eb-08111c108dda service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:49.285359 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1373/2732] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:22:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:22:49.949506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2f507-6337-4432-be92-ea877dc13185 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:49.949994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2f507-6337-4432-be92-ea877dc13185 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:49.950176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2f507-6337-4432-be92-ea877dc13185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:49.987455 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e2b2f507-6337-4432-be92-ea877dc13185 None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:22:49.988885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e2b2f507-6337-4432-be92-ea877dc13185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:50.060328 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1360/2733] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 9 13:22:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:22:50.083802 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cf3b5-4c26-4c32-86eb-08111c108dda service glance] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:22:50.085318 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-231cf3b5-4c26-4c32-86eb-08111c108dda service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:22:50.161207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cf3b5-4c26-4c32-86eb-08111c108dda service glance] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:22:50.166194 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f86ef-b458-4314-94ae-84c7ccdf616f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:22:50.166810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f86ef-b458-4314-94ae-84c7ccdf616f service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:22:50.167117 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f86ef-b458-4314-94ae-84c7ccdf616f service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:22:50.167395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f86ef-b458-4314-94ae-84c7ccdf616f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:22:50.175844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f86ef-b458-4314-94ae-84c7ccdf616f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:22:50.177433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f86ef-b458-4314-94ae-84c7ccdf616f service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:22:50.185386 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1374/2734] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:22:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:23:00.054705 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=53 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:00.182128 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=42 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:05.872196 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2f507-6337-4432-be92-ea877dc13185 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:05.936778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2f507-6337-4432-be92-ea877dc13185 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:05.942617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:05.943377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:05.943654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:05.943923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] PATH_INFO: `/v3/users/d90d7fe9cfc94440a603e6f010e10eb5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:05.945047 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d90d7fe9cfc94440a603e6f010e10eb5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:05.946977 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:05.996994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd90d7fe9cfc94440a603e6f010e10eb5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:06.004864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd90d7fe9cfc94440a603e6f010e10eb5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:06.017386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd90d7fe9cfc94440a603e6f010e10eb5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:06.018174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd90d7fe9cfc94440a603e6f010e10eb5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:06.025159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd90d7fe9cfc94440a603e6f010e10eb5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:06.031121 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1361/2735] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:05 2022] DELETE /identity/v3/users/d90d7fe9cfc94440a603e6f010e10eb5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:06.036369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b0b1f-1d15-4c89-8227-4589b56eaeae None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:06.036873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b0b1f-1d15-4c89-8227-4589b56eaeae None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:06.036993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b0b1f-1d15-4c89-8227-4589b56eaeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:06.055830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-246b0b1f-1d15-4c89-8227-4589b56eaeae None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:06.057040 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-246b0b1f-1d15-4c89-8227-4589b56eaeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:06.120631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1375/2736] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:06.131179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:06.132619 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:06.210413 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33c7bd-8d2b-4523-bda5-b73cf920888a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:06.214041 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677406a-1b42-4b68-88f6-3eab18f06c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:06.214645 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677406a-1b42-4b68-88f6-3eab18f06c34 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:06.214922 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677406a-1b42-4b68-88f6-3eab18f06c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:06.215224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677406a-1b42-4b68-88f6-3eab18f06c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:06.224421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677406a-1b42-4b68-88f6-3eab18f06c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:06.225854 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677406a-1b42-4b68-88f6-3eab18f06c34 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:06.235239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1362/2737] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:06.507170 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b0b1f-1d15-4c89-8227-4589b56eaeae None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:06.508984 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-246b0b1f-1d15-4c89-8227-4589b56eaeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:06.522271 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b0b1f-1d15-4c89-8227-4589b56eaeae None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:06.527754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:06.528407 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:06.528690 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:06.528967 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0 admin admin] PATH_INFO: `/v3/projects/6be36ce8b9db4257bfdfb671a298ae16` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:06.530106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be36ce8b9db4257bfdfb671a298ae16)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:06.532074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:06.594357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be36ce8b9db4257bfdfb671a298ae16', 'request_id': 'req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:06.602544 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be36ce8b9db4257bfdfb671a298ae16', 'request_id': 'req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:06.614660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be36ce8b9db4257bfdfb671a298ae16', 'request_id': 'req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:06.626994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0 admin admin] The token cache is being invalidate because project 6be36ce8b9db4257bfdfb671a298ae16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:06.627292 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:06.628043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e9fa1288-6388-40dd-9aa6-9d4674ace4b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:06.629473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1376/2738] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:06 2022] DELETE /identity/v3/projects/6be36ce8b9db4257bfdfb671a298ae16 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:06.634367 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b68f6-1f4f-4f91-b751-988cd02ff0eb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:06.634676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b68f6-1f4f-4f91-b751-988cd02ff0eb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:06.634951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b68f6-1f4f-4f91-b751-988cd02ff0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:06.643122 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:06.659420 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c35b68f6-1f4f-4f91-b751-988cd02ff0eb None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:06.660651 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c35b68f6-1f4f-4f91-b751-988cd02ff0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:06.722710 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1363/2739] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:06.731529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989747a-5a83-4dda-bd32-e9cff8a424c4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:06.731820 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989747a-5a83-4dda-bd32-e9cff8a424c4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:06.732096 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989747a-5a83-4dda-bd32-e9cff8a424c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:06.753532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b989747a-5a83-4dda-bd32-e9cff8a424c4 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:06.755027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b989747a-5a83-4dda-bd32-e9cff8a424c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:06.763507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1377/2740] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:06.768449 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b68f6-1f4f-4f91-b751-988cd02ff0eb None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:06.778158 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b68f6-1f4f-4f91-b751-988cd02ff0eb None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:06.783588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a49a4-28d9-4a4c-83f2-800146ef8387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:06.784190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a49a4-28d9-4a4c-83f2-800146ef8387 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:06.784471 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a49a4-28d9-4a4c-83f2-800146ef8387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:06.784746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a49a4-28d9-4a4c-83f2-800146ef8387 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:06.785398 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a49a4-28d9-4a4c-83f2-800146ef8387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:23:06.785705 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a49a4-28d9-4a4c-83f2-800146ef8387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:06.787340 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a49a4-28d9-4a4c-83f2-800146ef8387 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:06.799613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1364/2741] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:23:06.803907 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989747a-5a83-4dda-bd32-e9cff8a424c4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:06.813069 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989747a-5a83-4dda-bd32-e9cff8a424c4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:06.818585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e136fc2-71e6-47d6-b347-f0aa019b7a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:06.819231 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e136fc2-71e6-47d6-b347-f0aa019b7a78 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:06.819514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e136fc2-71e6-47d6-b347-f0aa019b7a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:06.819787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e136fc2-71e6-47d6-b347-f0aa019b7a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:06.820462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e136fc2-71e6-47d6-b347-f0aa019b7a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:06.822390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e136fc2-71e6-47d6-b347-f0aa019b7a78 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:06.849789 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1378/2742] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:23:06 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:06.853626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a49a4-28d9-4a4c-83f2-800146ef8387 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:06.864137 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a49a4-28d9-4a4c-83f2-800146ef8387 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:06.920261 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4e6c-1884-4956-b4fd-525e2f70242c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:06.920897 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4e6c-1884-4956-b4fd-525e2f70242c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:06.921060 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4e6c-1884-4956-b4fd-525e2f70242c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:06.921331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af4e6c-1884-4956-b4fd-525e2f70242c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:06.921962 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af4e6c-1884-4956-b4fd-525e2f70242c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:06.924718 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af4e6c-1884-4956-b4fd-525e2f70242c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:06.945071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1365/2743] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:23:06 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:06.949184 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e136fc2-71e6-47d6-b347-f0aa019b7a78 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:06.959378 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e136fc2-71e6-47d6-b347-f0aa019b7a78 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:06.963929 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae0293-0845-4f30-910a-9fda9d76f467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:06.964518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae0293-0845-4f30-910a-9fda9d76f467 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:06.964792 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae0293-0845-4f30-910a-9fda9d76f467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:06.965063 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae0293-0845-4f30-910a-9fda9d76f467 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:06.965692 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae0293-0845-4f30-910a-9fda9d76f467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:06.967302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae0293-0845-4f30-910a-9fda9d76f467 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:06.977555 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1379/2744] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:06.981424 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4e6c-1884-4956-b4fd-525e2f70242c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:06.992190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af4e6c-1884-4956-b4fd-525e2f70242c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:06.996800 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8139fb3-734a-4ab9-bd51-93c760ad913b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:06.997394 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8139fb3-734a-4ab9-bd51-93c760ad913b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:06.997674 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8139fb3-734a-4ab9-bd51-93c760ad913b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:06.997944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8139fb3-734a-4ab9-bd51-93c760ad913b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:06.998576 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8139fb3-734a-4ab9-bd51-93c760ad913b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:07.000452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8139fb3-734a-4ab9-bd51-93c760ad913b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:07.011500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1366/2745] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:07.015528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae0293-0845-4f30-910a-9fda9d76f467 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:07.025862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae0293-0845-4f30-910a-9fda9d76f467 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:07.030582 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7868e-0412-4b4f-b4b8-e9abe39ba864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:07.031268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7868e-0412-4b4f-b4b8-e9abe39ba864 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:07.031552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7868e-0412-4b4f-b4b8-e9abe39ba864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:07.031825 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7868e-0412-4b4f-b4b8-e9abe39ba864 admin admin] PATH_INFO: `/v3/projects/c37d9975f0964c1888eb9532cf404ab6/users/993410c5ed3f4030b6414c9af794cfa2/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:07.046778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7868e-0412-4b4f-b4b8-e9abe39ba864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37d9975f0964c1888eb9532cf404ab6, user_id=993410c5ed3f4030b6414c9af794cfa2, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:07.050146 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7868e-0412-4b4f-b4b8-e9abe39ba864 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:07.062682 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1380/2746] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:07 2022] PUT /identity/v3/projects/c37d9975f0964c1888eb9532cf404ab6/users/993410c5ed3f4030b6414c9af794cfa2/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:07.072960 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a393c-e453-4f91-b316-66f3044175e1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:07.073244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a393c-e453-4f91-b316-66f3044175e1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:07.073517 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a393c-e453-4f91-b316-66f3044175e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:07.101806 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a21a393c-e453-4f91-b316-66f3044175e1 None None] MFA Rules not processed for user `993410c5ed3f4030b6414c9af794cfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:07.103015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a21a393c-e453-4f91-b316-66f3044175e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:07.161479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:07.167954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1367/2747] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:07.178928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7868e-0412-4b4f-b4b8-e9abe39ba864 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:07.180627 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5ef7868e-0412-4b4f-b4b8-e9abe39ba864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:07.203336 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:07.259077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7868e-0412-4b4f-b4b8-e9abe39ba864 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:07.262756 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9920ed-f7e5-45f8-ae9b-bd446eadd1b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:07.263389 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9920ed-f7e5-45f8-ae9b-bd446eadd1b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:07.263669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9920ed-f7e5-45f8-ae9b-bd446eadd1b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:07.263942 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9920ed-f7e5-45f8-ae9b-bd446eadd1b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:07.272539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9920ed-f7e5-45f8-ae9b-bd446eadd1b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:07.273960 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9920ed-f7e5-45f8-ae9b-bd446eadd1b3 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:07.283211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1381/2748] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:23:07.306936 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a393c-e453-4f91-b316-66f3044175e1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:07.367941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a393c-e453-4f91-b316-66f3044175e1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:07.372503 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:07.373129 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:07.373410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:07.373685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] PATH_INFO: `/v3/users/993410c5ed3f4030b6414c9af794cfa2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:07.375077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=993410c5ed3f4030b6414c9af794cfa2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:07.377029 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:07.425170 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993410c5ed3f4030b6414c9af794cfa2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:07.433329 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993410c5ed3f4030b6414c9af794cfa2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:07.446415 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993410c5ed3f4030b6414c9af794cfa2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:07.447064 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993410c5ed3f4030b6414c9af794cfa2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:07.454083 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993410c5ed3f4030b6414c9af794cfa2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:07.460787 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1368/2749] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:07 2022] DELETE /identity/v3/users/993410c5ed3f4030b6414c9af794cfa2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:07.465430 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee582b2-51d6-4b83-acfa-2b38e5c25e7e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:07.465736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee582b2-51d6-4b83-acfa-2b38e5c25e7e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:07.466005 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee582b2-51d6-4b83-acfa-2b38e5c25e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:07.485587 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dee582b2-51d6-4b83-acfa-2b38e5c25e7e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:07.486782 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dee582b2-51d6-4b83-acfa-2b38e5c25e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:07.545537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1382/2750] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:07.555494 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:07.556885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:07.614469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad35f5-79e0-4516-ba37-e1151ee210ef admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:07.618149 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa746289-26ce-4262-84bf-2c64290e9522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:07.618755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa746289-26ce-4262-84bf-2c64290e9522 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:07.619034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa746289-26ce-4262-84bf-2c64290e9522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:07.619336 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa746289-26ce-4262-84bf-2c64290e9522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:07.629204 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa746289-26ce-4262-84bf-2c64290e9522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:07.630646 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa746289-26ce-4262-84bf-2c64290e9522 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:07.639959 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1369/2751] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:07.914766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee582b2-51d6-4b83-acfa-2b38e5c25e7e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:07.916219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dee582b2-51d6-4b83-acfa-2b38e5c25e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:07.929581 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee582b2-51d6-4b83-acfa-2b38e5c25e7e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:07.934173 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f4a84-6b76-4d23-9f17-5d4538143da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:07.934830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f4a84-6b76-4d23-9f17-5d4538143da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:07.935693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f4a84-6b76-4d23-9f17-5d4538143da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:07.935995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f4a84-6b76-4d23-9f17-5d4538143da0 admin admin] PATH_INFO: `/v3/projects/c37d9975f0964c1888eb9532cf404ab6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:07.937316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f4a84-6b76-4d23-9f17-5d4538143da0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c37d9975f0964c1888eb9532cf404ab6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:07.939274 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f4a84-6b76-4d23-9f17-5d4538143da0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:07.986859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-593f4a84-6b76-4d23-9f17-5d4538143da0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c37d9975f0964c1888eb9532cf404ab6', 'request_id': 'req-593f4a84-6b76-4d23-9f17-5d4538143da0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:07.994009 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-593f4a84-6b76-4d23-9f17-5d4538143da0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c37d9975f0964c1888eb9532cf404ab6', 'request_id': 'req-593f4a84-6b76-4d23-9f17-5d4538143da0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:07.999937 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-593f4a84-6b76-4d23-9f17-5d4538143da0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c37d9975f0964c1888eb9532cf404ab6', 'request_id': 'req-593f4a84-6b76-4d23-9f17-5d4538143da0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:08.011943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-593f4a84-6b76-4d23-9f17-5d4538143da0 admin admin] The token cache is being invalidate because project c37d9975f0964c1888eb9532cf404ab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:08.012227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-593f4a84-6b76-4d23-9f17-5d4538143da0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:08.012896 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-593f4a84-6b76-4d23-9f17-5d4538143da0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:08.014384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1383/2752] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:07 2022] DELETE /identity/v3/projects/c37d9975f0964c1888eb9532cf404ab6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:08.019903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff442a-e82d-45f9-824b-0641a030cece None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:08.020408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff442a-e82d-45f9-824b-0641a030cece None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:08.020524 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff442a-e82d-45f9-824b-0641a030cece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:08.040451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-09ff442a-e82d-45f9-824b-0641a030cece None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:08.041642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-09ff442a-e82d-45f9-824b-0641a030cece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:08.054316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:08.102033 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1370/2753] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:08.109829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4103249-3676-4b75-bf84-975c472e740c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:08.110310 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4103249-3676-4b75-bf84-975c472e740c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:08.110423 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4103249-3676-4b75-bf84-975c472e740c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:08.130269 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f4103249-3676-4b75-bf84-975c472e740c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:08.131509 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f4103249-3676-4b75-bf84-975c472e740c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:08.139338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1384/2754] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:08.144428 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff442a-e82d-45f9-824b-0641a030cece None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:08.153911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff442a-e82d-45f9-824b-0641a030cece None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:08.159397 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f1e28-0a60-4843-8073-2b4cafdb5863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:08.159993 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f1e28-0a60-4843-8073-2b4cafdb5863 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:08.160267 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f1e28-0a60-4843-8073-2b4cafdb5863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:08.160538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f1e28-0a60-4843-8073-2b4cafdb5863 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:08.161174 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f1e28-0a60-4843-8073-2b4cafdb5863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:23:08.161477 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f1e28-0a60-4843-8073-2b4cafdb5863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:08.163079 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f1e28-0a60-4843-8073-2b4cafdb5863 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:08.174617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1371/2755] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:23:08.178803 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4103249-3676-4b75-bf84-975c472e740c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:08.187851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4103249-3676-4b75-bf84-975c472e740c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:08.192754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87c1ff-bd5c-42de-bcf5-6d9dd84e6154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:08.193346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87c1ff-bd5c-42de-bcf5-6d9dd84e6154 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:08.193619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87c1ff-bd5c-42de-bcf5-6d9dd84e6154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:08.193888 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87c1ff-bd5c-42de-bcf5-6d9dd84e6154 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:08.194518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87c1ff-bd5c-42de-bcf5-6d9dd84e6154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:08.196448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87c1ff-bd5c-42de-bcf5-6d9dd84e6154 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:08.217727 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:08.230698 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1385/2756] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:23:08 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:08.231913 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f1e28-0a60-4843-8073-2b4cafdb5863 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:08.243577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f1e28-0a60-4843-8073-2b4cafdb5863 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:08.299864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433cb04-adb4-47ae-9b12-eb7d777991e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:08.300453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433cb04-adb4-47ae-9b12-eb7d777991e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:08.300729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433cb04-adb4-47ae-9b12-eb7d777991e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:08.301004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433cb04-adb4-47ae-9b12-eb7d777991e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:08.301614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433cb04-adb4-47ae-9b12-eb7d777991e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:08.303542 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433cb04-adb4-47ae-9b12-eb7d777991e2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:08.324283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1372/2757] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:23:08 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:08.328510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87c1ff-bd5c-42de-bcf5-6d9dd84e6154 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:08.339190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87c1ff-bd5c-42de-bcf5-6d9dd84e6154 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:08.343839 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3714a5-d717-4d7d-a289-fc44419a0041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:08.344441 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3714a5-d717-4d7d-a289-fc44419a0041 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:08.344740 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3714a5-d717-4d7d-a289-fc44419a0041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:08.345527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3714a5-d717-4d7d-a289-fc44419a0041 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:08.346357 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3714a5-d717-4d7d-a289-fc44419a0041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:08.347985 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3714a5-d717-4d7d-a289-fc44419a0041 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:08.357827 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1386/2758] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:08.361858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433cb04-adb4-47ae-9b12-eb7d777991e2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:08.371527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433cb04-adb4-47ae-9b12-eb7d777991e2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:08.377130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c691a09-b745-4fed-9caa-69a1646b459b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:08.377729 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c691a09-b745-4fed-9caa-69a1646b459b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:08.378004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c691a09-b745-4fed-9caa-69a1646b459b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:08.378270 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c691a09-b745-4fed-9caa-69a1646b459b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:08.378900 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c691a09-b745-4fed-9caa-69a1646b459b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:08.380513 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c691a09-b745-4fed-9caa-69a1646b459b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:08.390783 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1373/2759] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:08.395401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3714a5-d717-4d7d-a289-fc44419a0041 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:08.405198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3714a5-d717-4d7d-a289-fc44419a0041 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:08.409955 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dd87b-9cae-4ebc-ae46-6412448ea4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:08.410589 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dd87b-9cae-4ebc-ae46-6412448ea4c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:08.410867 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dd87b-9cae-4ebc-ae46-6412448ea4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:08.411178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dd87b-9cae-4ebc-ae46-6412448ea4c0 admin admin] PATH_INFO: `/v3/projects/8c43be761bbc43e5a44d10c845d5e32f/users/a5fe2780d3804fd59568e72fd0f07e09/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:08.425620 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dd87b-9cae-4ebc-ae46-6412448ea4c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c43be761bbc43e5a44d10c845d5e32f, user_id=a5fe2780d3804fd59568e72fd0f07e09, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:08.428313 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dd87b-9cae-4ebc-ae46-6412448ea4c0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:08.441542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1387/2760] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:08 2022] PUT /identity/v3/projects/8c43be761bbc43e5a44d10c845d5e32f/users/a5fe2780d3804fd59568e72fd0f07e09/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:23:08.507156 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e846e7-5017-44fc-824b-b70fe72dc942 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:08.507376 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e846e7-5017-44fc-824b-b70fe72dc942 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:08.507632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e846e7-5017-44fc-824b-b70fe72dc942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:08.537022 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-58e846e7-5017-44fc-824b-b70fe72dc942 None None] MFA Rules not processed for user `a5fe2780d3804fd59568e72fd0f07e09`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:08.538242 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-58e846e7-5017-44fc-824b-b70fe72dc942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:08.601240 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1374/2761] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:08.612723 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dd87b-9cae-4ebc-ae46-6412448ea4c0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:08.614195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-645dd87b-9cae-4ebc-ae46-6412448ea4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:08.671907 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dd87b-9cae-4ebc-ae46-6412448ea4c0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:08.675790 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b3047-36e7-404f-88cf-29a42dcb31d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:08.676389 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b3047-36e7-404f-88cf-29a42dcb31d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:08.676664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b3047-36e7-404f-88cf-29a42dcb31d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:08.676930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b3047-36e7-404f-88cf-29a42dcb31d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:08.684992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b3047-36e7-404f-88cf-29a42dcb31d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:08.686400 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b3047-36e7-404f-88cf-29a42dcb31d6 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:08.695810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1388/2762] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:18.586400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e846e7-5017-44fc-824b-b70fe72dc942 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:18.604056 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:18.651685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e846e7-5017-44fc-824b-b70fe72dc942 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:18.656167 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:18.656799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:18.657082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:18.657425 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] PATH_INFO: `/v3/users/a5fe2780d3804fd59568e72fd0f07e09` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:18.658558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5fe2780d3804fd59568e72fd0f07e09)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:18.660508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:18.691899 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:18.707286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5fe2780d3804fd59568e72fd0f07e09'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:18.714568 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5fe2780d3804fd59568e72fd0f07e09'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:18.726654 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5fe2780d3804fd59568e72fd0f07e09'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:18.727321 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5fe2780d3804fd59568e72fd0f07e09'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:18.734180 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5fe2780d3804fd59568e72fd0f07e09'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:18.740772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1375/2763] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:18 2022] DELETE /identity/v3/users/a5fe2780d3804fd59568e72fd0f07e09 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:18.745223 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c24d00-cd0b-476a-8187-57283537b9ab None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:18.745529 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c24d00-cd0b-476a-8187-57283537b9ab None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:18.745802 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c24d00-cd0b-476a-8187-57283537b9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:18.765866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-86c24d00-cd0b-476a-8187-57283537b9ab None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:18.767156 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-86c24d00-cd0b-476a-8187-57283537b9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:18.825834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1389/2764] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:23:18.835612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:18.837022 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:18.893245 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ed202-f88d-4fa9-9374-22d7e2e42356 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:18.896904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f06e6-e91a-4e25-87e7-fe5c1a34d6e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:18.897529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f06e6-e91a-4e25-87e7-fe5c1a34d6e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:18.897789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f06e6-e91a-4e25-87e7-fe5c1a34d6e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:18.898061 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f06e6-e91a-4e25-87e7-fe5c1a34d6e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:18.907887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f06e6-e91a-4e25-87e7-fe5c1a34d6e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:18.909323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f06e6-e91a-4e25-87e7-fe5c1a34d6e6 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:18.918681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1376/2765] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:19.177666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c24d00-cd0b-476a-8187-57283537b9ab None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:19.179384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-86c24d00-cd0b-476a-8187-57283537b9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:19.191892 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c24d00-cd0b-476a-8187-57283537b9ab None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:19.196748 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:19.197380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:19.197658 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:19.197927 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] PATH_INFO: `/v3/projects/8c43be761bbc43e5a44d10c845d5e32f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:19.199114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c43be761bbc43e5a44d10c845d5e32f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:19.201026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:19.251594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c43be761bbc43e5a44d10c845d5e32f', 'request_id': 'req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:19.258851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c43be761bbc43e5a44d10c845d5e32f', 'request_id': 'req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:19.264418 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c43be761bbc43e5a44d10c845d5e32f', 'request_id': 'req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:19.275924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] The token cache is being invalidate because project 8c43be761bbc43e5a44d10c845d5e32f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:19.276208 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:19.276830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:19.278324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1390/2766] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:19 2022] DELETE /identity/v3/projects/8c43be761bbc43e5a44d10c845d5e32f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:27.927623 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f06e6-e91a-4e25-87e7-fe5c1a34d6e6 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:27.931246 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2e6f06e6-e91a-4e25-87e7-fe5c1a34d6e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:27.995876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:28.067004 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f06e6-e91a-4e25-87e7-fe5c1a34d6e6 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:28.258148 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3324a79-cefc-419b-bd10-38f9eb2f633c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:28.258802 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3324a79-cefc-419b-bd10-38f9eb2f633c service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:28.259160 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3324a79-cefc-419b-bd10-38f9eb2f633c service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:28.259434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3324a79-cefc-419b-bd10-38f9eb2f633c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:28.261171 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b3324a79-cefc-419b-bd10-38f9eb2f633c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:28.273131 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3324a79-cefc-419b-bd10-38f9eb2f633c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:28.274703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3324a79-cefc-419b-bd10-38f9eb2f633c service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:28.285835 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1377/2767] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:29.272913 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:32.176361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:32.178095 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:32.236458 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1d78c-0ef7-4497-95e3-49ebe6032a34 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:32.240114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21372e-8e30-4c53-9af7-ba06dcd27154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:32.240713 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21372e-8e30-4c53-9af7-ba06dcd27154 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:32.240994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21372e-8e30-4c53-9af7-ba06dcd27154 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:32.241268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21372e-8e30-4c53-9af7-ba06dcd27154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:32.243001 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9a21372e-8e30-4c53-9af7-ba06dcd27154 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:32.305536 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21372e-8e30-4c53-9af7-ba06dcd27154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:32.306974 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21372e-8e30-4c53-9af7-ba06dcd27154 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:32.316826 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1391/2768] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:35.424497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3324a79-cefc-419b-bd10-38f9eb2f633c service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:35.425968 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b3324a79-cefc-419b-bd10-38f9eb2f633c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:35.446194 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3324a79-cefc-419b-bd10-38f9eb2f633c service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:35.451019 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0b26e-c47c-4222-9aed-94e8ec1982ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:35.451796 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0b26e-c47c-4222-9aed-94e8ec1982ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:35.452152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0b26e-c47c-4222-9aed-94e8ec1982ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:35.452451 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0b26e-c47c-4222-9aed-94e8ec1982ce admin admin] PATH_INFO: `/v3/users/ca9eb6b9dc6e409a954c7993ebe5a5c3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:35.453624 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0b26e-c47c-4222-9aed-94e8ec1982ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9eb6b9dc6e409a954c7993ebe5a5c3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:35.456141 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0b26e-c47c-4222-9aed-94e8ec1982ce admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:35.507023 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-57b0b26e-c47c-4222-9aed-94e8ec1982ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9eb6b9dc6e409a954c7993ebe5a5c3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:35.514538 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-57b0b26e-c47c-4222-9aed-94e8ec1982ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9eb6b9dc6e409a954c7993ebe5a5c3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:35.527118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-57b0b26e-c47c-4222-9aed-94e8ec1982ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9eb6b9dc6e409a954c7993ebe5a5c3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:35.527744 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-57b0b26e-c47c-4222-9aed-94e8ec1982ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9eb6b9dc6e409a954c7993ebe5a5c3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:35.534715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-57b0b26e-c47c-4222-9aed-94e8ec1982ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9eb6b9dc6e409a954c7993ebe5a5c3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:35.540543 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1378/2769] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:35 2022] DELETE /identity/v3/users/ca9eb6b9dc6e409a954c7993ebe5a5c3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:35.711764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21372e-8e30-4c53-9af7-ba06dcd27154 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:35.722017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21372e-8e30-4c53-9af7-ba06dcd27154 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:35.780214 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270be0af-6fd9-4d22-81f9-5f4903e6acfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:35.780850 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270be0af-6fd9-4d22-81f9-5f4903e6acfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:35.781137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270be0af-6fd9-4d22-81f9-5f4903e6acfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:35.781414 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270be0af-6fd9-4d22-81f9-5f4903e6acfd admin admin] PATH_INFO: `/v3/projects/92f354242a7b4ae4b774681eb633c2b2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:35.782634 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270be0af-6fd9-4d22-81f9-5f4903e6acfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=92f354242a7b4ae4b774681eb633c2b2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:35.784586 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270be0af-6fd9-4d22-81f9-5f4903e6acfd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:35.832618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-270be0af-6fd9-4d22-81f9-5f4903e6acfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92f354242a7b4ae4b774681eb633c2b2', 'request_id': 'req-270be0af-6fd9-4d22-81f9-5f4903e6acfd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:35.835246 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:35.840625 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-270be0af-6fd9-4d22-81f9-5f4903e6acfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92f354242a7b4ae4b774681eb633c2b2', 'request_id': 'req-270be0af-6fd9-4d22-81f9-5f4903e6acfd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:35.847329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-270be0af-6fd9-4d22-81f9-5f4903e6acfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92f354242a7b4ae4b774681eb633c2b2', 'request_id': 'req-270be0af-6fd9-4d22-81f9-5f4903e6acfd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:35.858208 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-270be0af-6fd9-4d22-81f9-5f4903e6acfd admin admin] The token cache is being invalidate because project 92f354242a7b4ae4b774681eb633c2b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:35.858488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-270be0af-6fd9-4d22-81f9-5f4903e6acfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:35.859198 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-270be0af-6fd9-4d22-81f9-5f4903e6acfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:35.861297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1392/2770] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:35 2022] DELETE /identity/v3/projects/92f354242a7b4ae4b774681eb633c2b2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:35.866161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f746f53-6d0e-44f1-bd23-647c7eceb8e4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:35.866514 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f746f53-6d0e-44f1-bd23-647c7eceb8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:35.866881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f746f53-6d0e-44f1-bd23-647c7eceb8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:35.888149 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0f746f53-6d0e-44f1-bd23-647c7eceb8e4 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:35.889366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0f746f53-6d0e-44f1-bd23-647c7eceb8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:35.909049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:35.954901 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1379/2771] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:35.962800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a8262-a434-4cbb-b1e2-4916e196aa79 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:35.963111 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a8262-a434-4cbb-b1e2-4916e196aa79 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:35.963386 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a8262-a434-4cbb-b1e2-4916e196aa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:35.983382 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ed4a8262-a434-4cbb-b1e2-4916e196aa79 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:35.984580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ed4a8262-a434-4cbb-b1e2-4916e196aa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:35.993428 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1393/2772] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:35.997591 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f746f53-6d0e-44f1-bd23-647c7eceb8e4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:36.008622 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f746f53-6d0e-44f1-bd23-647c7eceb8e4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:36.013253 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8ec6e-7172-426a-bb57-5be7499db2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:36.013863 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8ec6e-7172-426a-bb57-5be7499db2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:36.014139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8ec6e-7172-426a-bb57-5be7499db2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:36.014407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8ec6e-7172-426a-bb57-5be7499db2c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:36.015101 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8ec6e-7172-426a-bb57-5be7499db2c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:23:36.015406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8ec6e-7172-426a-bb57-5be7499db2c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:36.017024 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8ec6e-7172-426a-bb57-5be7499db2c3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:36.028430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1380/2773] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:23:36.032714 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a8262-a434-4cbb-b1e2-4916e196aa79 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:36.041731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a8262-a434-4cbb-b1e2-4916e196aa79 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:36.046319 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48245109-ca54-48e4-9fe1-c7c872c3726c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:36.046909 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48245109-ca54-48e4-9fe1-c7c872c3726c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:36.047224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48245109-ca54-48e4-9fe1-c7c872c3726c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:36.048041 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48245109-ca54-48e4-9fe1-c7c872c3726c admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:36.048850 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48245109-ca54-48e4-9fe1-c7c872c3726c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:36.050745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48245109-ca54-48e4-9fe1-c7c872c3726c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:36.077528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1394/2774] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:23:36 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:36.081643 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8ec6e-7172-426a-bb57-5be7499db2c3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:36.091509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8ec6e-7172-426a-bb57-5be7499db2c3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:36.148053 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98359f-ebbb-4dc6-8b70-6ad4d44d447d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:36.148655 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98359f-ebbb-4dc6-8b70-6ad4d44d447d admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:36.148931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98359f-ebbb-4dc6-8b70-6ad4d44d447d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:36.149199 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98359f-ebbb-4dc6-8b70-6ad4d44d447d admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:36.149816 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98359f-ebbb-4dc6-8b70-6ad4d44d447d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:36.151768 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98359f-ebbb-4dc6-8b70-6ad4d44d447d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:36.173130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1381/2775] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:23:36 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:36.177161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48245109-ca54-48e4-9fe1-c7c872c3726c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:36.187430 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48245109-ca54-48e4-9fe1-c7c872c3726c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:36.192117 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfece6-b1b3-4255-9814-e75090479995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:36.192716 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfece6-b1b3-4255-9814-e75090479995 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:36.192994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfece6-b1b3-4255-9814-e75090479995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:36.193266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfece6-b1b3-4255-9814-e75090479995 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:36.193950 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfece6-b1b3-4255-9814-e75090479995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:36.195594 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfece6-b1b3-4255-9814-e75090479995 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:36.205463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1395/2776] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:36.209362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98359f-ebbb-4dc6-8b70-6ad4d44d447d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:36.219128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98359f-ebbb-4dc6-8b70-6ad4d44d447d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:36.224270 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8133b8-d109-4524-b203-62768464fb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:36.224856 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8133b8-d109-4524-b203-62768464fb0a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:36.225130 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8133b8-d109-4524-b203-62768464fb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:36.225423 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8133b8-d109-4524-b203-62768464fb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:36.226049 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8133b8-d109-4524-b203-62768464fb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:36.227753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8133b8-d109-4524-b203-62768464fb0a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:36.238767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1382/2777] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:36.243025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfece6-b1b3-4255-9814-e75090479995 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:36.253761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfece6-b1b3-4255-9814-e75090479995 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:36.258467 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ed91e-2c06-42dd-8e41-68cdab1fdd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:36.259153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ed91e-2c06-42dd-8e41-68cdab1fdd31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:36.259461 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ed91e-2c06-42dd-8e41-68cdab1fdd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:36.259736 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ed91e-2c06-42dd-8e41-68cdab1fdd31 admin admin] PATH_INFO: `/v3/projects/5c87ad71868b48de9b0e919dc63b0716/users/c93f13265b524f43be4f2147e6686a55/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:36.274779 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ed91e-2c06-42dd-8e41-68cdab1fdd31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c87ad71868b48de9b0e919dc63b0716, user_id=c93f13265b524f43be4f2147e6686a55, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:36.277443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ed91e-2c06-42dd-8e41-68cdab1fdd31 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:36.290854 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1396/2778] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:36 2022] PUT /identity/v3/projects/5c87ad71868b48de9b0e919dc63b0716/users/c93f13265b524f43be4f2147e6686a55/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:23:36.302726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7a6d2-ee5a-47c3-bd61-b999395b8b90 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:36.303013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7a6d2-ee5a-47c3-bd61-b999395b8b90 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:36.303323 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7a6d2-ee5a-47c3-bd61-b999395b8b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:36.330128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-aeb7a6d2-ee5a-47c3-bd61-b999395b8b90 None None] MFA Rules not processed for user `c93f13265b524f43be4f2147e6686a55`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:36.331379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aeb7a6d2-ee5a-47c3-bd61-b999395b8b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:36.393391 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1383/2779] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:36.404087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ed91e-2c06-42dd-8e41-68cdab1fdd31 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:36.405815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1f0ed91e-2c06-42dd-8e41-68cdab1fdd31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:36.462631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ed91e-2c06-42dd-8e41-68cdab1fdd31 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:36.466145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80316be-d797-4acf-a787-88c1eb2028cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:36.466744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80316be-d797-4acf-a787-88c1eb2028cd service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:36.467017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80316be-d797-4acf-a787-88c1eb2028cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:36.467316 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80316be-d797-4acf-a787-88c1eb2028cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:36.475939 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80316be-d797-4acf-a787-88c1eb2028cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:36.477361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80316be-d797-4acf-a787-88c1eb2028cd service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:36.486673 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1397/2780] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:40.948142 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7a6d2-ee5a-47c3-bd61-b999395b8b90 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:40.949544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-aeb7a6d2-ee5a-47c3-bd61-b999395b8b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:40.973788 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:41.014709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7a6d2-ee5a-47c3-bd61-b999395b8b90 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:41.018950 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df865e-cfa9-4a25-b88e-b8ee82e804c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:41.019601 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df865e-cfa9-4a25-b88e-b8ee82e804c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:41.019885 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df865e-cfa9-4a25-b88e-b8ee82e804c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:41.020161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df865e-cfa9-4a25-b88e-b8ee82e804c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:41.021899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-33df865e-cfa9-4a25-b88e-b8ee82e804c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:41.083294 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df865e-cfa9-4a25-b88e-b8ee82e804c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:41.084767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df865e-cfa9-4a25-b88e-b8ee82e804c5 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:41.093797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1384/2781] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:23:44.883302 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80316be-d797-4acf-a787-88c1eb2028cd service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:44.947612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80316be-d797-4acf-a787-88c1eb2028cd service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:44.952200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:44.952829 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:44.953108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:44.953377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] PATH_INFO: `/v3/users/c93f13265b524f43be4f2147e6686a55` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:44.955028 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93f13265b524f43be4f2147e6686a55)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:44.956964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:44.978451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:45.004097 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93f13265b524f43be4f2147e6686a55'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:45.011209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93f13265b524f43be4f2147e6686a55'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:45.011728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93f13265b524f43be4f2147e6686a55'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:45.024350 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93f13265b524f43be4f2147e6686a55'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:45.031324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93f13265b524f43be4f2147e6686a55'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:45.037947 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1398/2782] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:44 2022] DELETE /identity/v3/users/c93f13265b524f43be4f2147e6686a55 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:45.042394 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d86de9-4425-47da-a29b-785bf384d473 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:45.042891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d86de9-4425-47da-a29b-785bf384d473 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:45.043001 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d86de9-4425-47da-a29b-785bf384d473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:45.061849 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b2d86de9-4425-47da-a29b-785bf384d473 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:45.063081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b2d86de9-4425-47da-a29b-785bf384d473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:45.122150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1385/2783] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:45.133694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:45.135153 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:45.195451 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b9119-ff9b-413b-a189-39a68b306edb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:45.199147 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd427021-fb7f-4315-b16a-7de3a19d7121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:45.199765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd427021-fb7f-4315-b16a-7de3a19d7121 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:45.200043 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd427021-fb7f-4315-b16a-7de3a19d7121 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:45.200315 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd427021-fb7f-4315-b16a-7de3a19d7121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:45.210433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd427021-fb7f-4315-b16a-7de3a19d7121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:45.211894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd427021-fb7f-4315-b16a-7de3a19d7121 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:45.221932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1399/2784] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:45.487998 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d86de9-4425-47da-a29b-785bf384d473 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:45.489440 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b2d86de9-4425-47da-a29b-785bf384d473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:45.501487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d86de9-4425-47da-a29b-785bf384d473 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:45.506100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc60f99-c62a-427b-a990-7783d2704669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:45.506736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc60f99-c62a-427b-a990-7783d2704669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:45.507016 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc60f99-c62a-427b-a990-7783d2704669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:45.507333 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc60f99-c62a-427b-a990-7783d2704669 admin admin] PATH_INFO: `/v3/projects/5c87ad71868b48de9b0e919dc63b0716` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:45.508444 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc60f99-c62a-427b-a990-7783d2704669 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c87ad71868b48de9b0e919dc63b0716)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:45.510353 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc60f99-c62a-427b-a990-7783d2704669 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:45.556882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0dc60f99-c62a-427b-a990-7783d2704669 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c87ad71868b48de9b0e919dc63b0716', 'request_id': 'req-0dc60f99-c62a-427b-a990-7783d2704669'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:45.562232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0dc60f99-c62a-427b-a990-7783d2704669 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c87ad71868b48de9b0e919dc63b0716', 'request_id': 'req-0dc60f99-c62a-427b-a990-7783d2704669'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:45.568432 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0dc60f99-c62a-427b-a990-7783d2704669 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c87ad71868b48de9b0e919dc63b0716', 'request_id': 'req-0dc60f99-c62a-427b-a990-7783d2704669'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:45.571995 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:45.583923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0dc60f99-c62a-427b-a990-7783d2704669 admin admin] The token cache is being invalidate because project 5c87ad71868b48de9b0e919dc63b0716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:45.584206 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0dc60f99-c62a-427b-a990-7783d2704669 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:45.584817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0dc60f99-c62a-427b-a990-7783d2704669 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:45.586326 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1386/2785] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:45 2022] DELETE /identity/v3/projects/5c87ad71868b48de9b0e919dc63b0716 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:45.592075 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabd98-7ea6-4653-95c0-a02d4ce731a4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:45.592361 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabd98-7ea6-4653-95c0-a02d4ce731a4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:45.592654 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabd98-7ea6-4653-95c0-a02d4ce731a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:45.613632 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d0dabd98-7ea6-4653-95c0-a02d4ce731a4 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:45.614865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d0dabd98-7ea6-4653-95c0-a02d4ce731a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:45.684056 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1400/2786] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:45.692719 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cae8d0-33e1-4700-aa17-52c663ced6ed None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:45.693002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cae8d0-33e1-4700-aa17-52c663ced6ed None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:45.693272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cae8d0-33e1-4700-aa17-52c663ced6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:45.713428 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f2cae8d0-33e1-4700-aa17-52c663ced6ed None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:45.714723 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f2cae8d0-33e1-4700-aa17-52c663ced6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:45.723610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1387/2787] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:45.727878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dabd98-7ea6-4653-95c0-a02d4ce731a4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:45.737746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dabd98-7ea6-4653-95c0-a02d4ce731a4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:45.742303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aadad4-4d01-4fec-8ab0-bff7e3866be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:45.742911 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aadad4-4d01-4fec-8ab0-bff7e3866be8 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:45.743223 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aadad4-4d01-4fec-8ab0-bff7e3866be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:45.743496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aadad4-4d01-4fec-8ab0-bff7e3866be8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:45.744141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aadad4-4d01-4fec-8ab0-bff7e3866be8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:23:45.744443 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aadad4-4d01-4fec-8ab0-bff7e3866be8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:45.746064 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aadad4-4d01-4fec-8ab0-bff7e3866be8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:45.760518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1401/2788] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:23:45.764465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cae8d0-33e1-4700-aa17-52c663ced6ed None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:45.774100 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cae8d0-33e1-4700-aa17-52c663ced6ed None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:45.779351 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a102d5-943b-43d3-b83d-0d8af323919f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:45.779939 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a102d5-943b-43d3-b83d-0d8af323919f admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:45.780215 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a102d5-943b-43d3-b83d-0d8af323919f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:45.780484 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a102d5-943b-43d3-b83d-0d8af323919f admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:45.781116 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a102d5-943b-43d3-b83d-0d8af323919f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:45.783736 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a102d5-943b-43d3-b83d-0d8af323919f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:45.811188 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1388/2789] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:23:45 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:45.816136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aadad4-4d01-4fec-8ab0-bff7e3866be8 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:45.826760 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aadad4-4d01-4fec-8ab0-bff7e3866be8 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:45.861597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:45.874163 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a102d5-943b-43d3-b83d-0d8af323919f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:45.876712 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-28a102d5-943b-43d3-b83d-0d8af323919f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:45.900778 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4aaa7-e139-462c-98cf-dc1d2df16dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:45.901393 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4aaa7-e139-462c-98cf-dc1d2df16dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:45.901669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4aaa7-e139-462c-98cf-dc1d2df16dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:45.901943 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4aaa7-e139-462c-98cf-dc1d2df16dd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:45.902571 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4aaa7-e139-462c-98cf-dc1d2df16dd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:45.904534 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4aaa7-e139-462c-98cf-dc1d2df16dd7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:45.927088 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1402/2790] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:23:45 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:45.929958 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4aaa7-e139-462c-98cf-dc1d2df16dd7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:45.939907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a102d5-943b-43d3-b83d-0d8af323919f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:45.940100 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4aaa7-e139-462c-98cf-dc1d2df16dd7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:45.943540 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de2ef2-95a7-44ca-9f40-5013312608c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:45.944176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de2ef2-95a7-44ca-9f40-5013312608c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:45.944509 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de2ef2-95a7-44ca-9f40-5013312608c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:45.944637 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bd3ab-d3ba-40be-80e6-93505e363d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:45.944785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de2ef2-95a7-44ca-9f40-5013312608c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:45.945190 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bd3ab-d3ba-40be-80e6-93505e363d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:45.945471 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bd3ab-d3ba-40be-80e6-93505e363d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:45.945746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bd3ab-d3ba-40be-80e6-93505e363d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:45.946381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94bd3ab-d3ba-40be-80e6-93505e363d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:45.946599 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d9de2ef2-95a7-44ca-9f40-5013312608c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:45.948012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94bd3ab-d3ba-40be-80e6-93505e363d83 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:45.959939 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1403/2791] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:23:45.960175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de2ef2-95a7-44ca-9f40-5013312608c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:45.961443 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de2ef2-95a7-44ca-9f40-5013312608c6 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:45.963831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bd3ab-d3ba-40be-80e6-93505e363d83 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:45.971660 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1389/2792] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:45.975261 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bd3ab-d3ba-40be-80e6-93505e363d83 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:45.980123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e5fd2-a1c0-4a3e-a765-0150953ee919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:45.980756 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e5fd2-a1c0-4a3e-a765-0150953ee919 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:45.981057 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e5fd2-a1c0-4a3e-a765-0150953ee919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:45.981330 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e5fd2-a1c0-4a3e-a765-0150953ee919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:45.981971 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e5fd2-a1c0-4a3e-a765-0150953ee919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:45.983603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e5fd2-a1c0-4a3e-a765-0150953ee919 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:45.994109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1404/2793] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:23:45.998277 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de2ef2-95a7-44ca-9f40-5013312608c6 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:46.009082 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de2ef2-95a7-44ca-9f40-5013312608c6 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:46.013656 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0c280-a1c3-4963-b23d-e9849d97e594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:46.014296 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0c280-a1c3-4963-b23d-e9849d97e594 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:46.014570 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0c280-a1c3-4963-b23d-e9849d97e594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:46.014844 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0c280-a1c3-4963-b23d-e9849d97e594 admin admin] PATH_INFO: `/v3/projects/7104f49476a840388aecea7a81b1ad10/users/8a786c7207204ea78d931db647a1e2eb/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:46.030407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0c280-a1c3-4963-b23d-e9849d97e594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7104f49476a840388aecea7a81b1ad10, user_id=8a786c7207204ea78d931db647a1e2eb, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:46.033375 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0c280-a1c3-4963-b23d-e9849d97e594 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:46.046622 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1390/2794] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:45 2022] PUT /identity/v3/projects/7104f49476a840388aecea7a81b1ad10/users/8a786c7207204ea78d931db647a1e2eb/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:46.058940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a5932-85c6-4f32-aac9-2e489d033049 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:46.059390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a5932-85c6-4f32-aac9-2e489d033049 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:46.059739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a5932-85c6-4f32-aac9-2e489d033049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:46.090505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-919a5932-85c6-4f32-aac9-2e489d033049 None None] MFA Rules not processed for user `8a786c7207204ea78d931db647a1e2eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:46.091777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-919a5932-85c6-4f32-aac9-2e489d033049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:46.160200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1405/2795] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:46.164586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0c280-a1c3-4963-b23d-e9849d97e594 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:46.228662 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0c280-a1c3-4963-b23d-e9849d97e594 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:46.233366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569d558-7952-4c4e-98e9-29c1f94f40c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:46.233978 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569d558-7952-4c4e-98e9-29c1f94f40c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:46.234254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569d558-7952-4c4e-98e9-29c1f94f40c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:46.234521 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569d558-7952-4c4e-98e9-29c1f94f40c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:46.235703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569d558-7952-4c4e-98e9-29c1f94f40c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:46.237590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569d558-7952-4c4e-98e9-29c1f94f40c3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:46.268378 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1391/2796] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:23:46 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:46.269556 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a5932-85c6-4f32-aac9-2e489d033049 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:46.280238 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a5932-85c6-4f32-aac9-2e489d033049 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:46.341054 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5fc36-fb75-45cc-9794-77b0d25d71c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:46.341670 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5fc36-fb75-45cc-9794-77b0d25d71c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:46.341951 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5fc36-fb75-45cc-9794-77b0d25d71c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:46.342227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5fc36-fb75-45cc-9794-77b0d25d71c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:46.342866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5fc36-fb75-45cc-9794-77b0d25d71c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:46.344837 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5fc36-fb75-45cc-9794-77b0d25d71c4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:46.371223 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1406/2797] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:23:46 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:46.372536 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569d558-7952-4c4e-98e9-29c1f94f40c3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:46.384190 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569d558-7952-4c4e-98e9-29c1f94f40c3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:46.389022 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf7a71-4c33-40bd-a8d8-7d91c8acca82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:46.389632 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf7a71-4c33-40bd-a8d8-7d91c8acca82 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:46.389908 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf7a71-4c33-40bd-a8d8-7d91c8acca82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:46.390176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf7a71-4c33-40bd-a8d8-7d91c8acca82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:46.390815 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf7a71-4c33-40bd-a8d8-7d91c8acca82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:46.392531 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf7a71-4c33-40bd-a8d8-7d91c8acca82 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:46.403200 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1392/2798] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:46.407543 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5fc36-fb75-45cc-9794-77b0d25d71c4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:46.417725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5fc36-fb75-45cc-9794-77b0d25d71c4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:46.422994 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9d427-b849-4e5a-b531-59124ee90a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:46.423955 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9d427-b849-4e5a-b531-59124ee90a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:46.424145 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9d427-b849-4e5a-b531-59124ee90a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:46.424356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9d427-b849-4e5a-b531-59124ee90a73 admin admin] PATH_INFO: `/v3/projects/7b0aba874ec5436b89199bb4469ccf88/users/b82adabf2646455597fc5802a2ea3819/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:46.437662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:46.442162 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9d427-b849-4e5a-b531-59124ee90a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0aba874ec5436b89199bb4469ccf88, user_id=b82adabf2646455597fc5802a2ea3819, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:46.445731 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9d427-b849-4e5a-b531-59124ee90a73 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:46.461455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1407/2799] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:46 2022] PUT /identity/v3/projects/7b0aba874ec5436b89199bb4469ccf88/users/b82adabf2646455597fc5802a2ea3819/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:46.465404 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf7a71-4c33-40bd-a8d8-7d91c8acca82 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:46.475640 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf7a71-4c33-40bd-a8d8-7d91c8acca82 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:46.492735 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:46.535465 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468c6bc-338e-4451-abc7-2057439f17ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:46.536322 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468c6bc-338e-4451-abc7-2057439f17ae admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:46.536763 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468c6bc-338e-4451-abc7-2057439f17ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:46.537155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468c6bc-338e-4451-abc7-2057439f17ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:46.537865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468c6bc-338e-4451-abc7-2057439f17ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:46.539455 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468c6bc-338e-4451-abc7-2057439f17ae admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:46.553668 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1393/2800] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:46.554893 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9d427-b849-4e5a-b531-59124ee90a73 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:46.565821 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9d427-b849-4e5a-b531-59124ee90a73 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:46.570672 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bef8e5-1a97-4a0a-886d-0ce945d239c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:46.571359 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bef8e5-1a97-4a0a-886d-0ce945d239c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:46.571643 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bef8e5-1a97-4a0a-886d-0ce945d239c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:46.571921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bef8e5-1a97-4a0a-886d-0ce945d239c2 admin admin] PATH_INFO: `/v3/projects/7b0aba874ec5436b89199bb4469ccf88/users/b82adabf2646455597fc5802a2ea3819/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:46.574091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bef8e5-1a97-4a0a-886d-0ce945d239c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0aba874ec5436b89199bb4469ccf88, user_id=b82adabf2646455597fc5802a2ea3819, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:46.576767 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bef8e5-1a97-4a0a-886d-0ce945d239c2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:46.587832 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1408/2801] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:46 2022] PUT /identity/v3/projects/7b0aba874ec5436b89199bb4469ccf88/users/b82adabf2646455597fc5802a2ea3819/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:46.598954 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d3994-681d-4e4e-a0d2-a2f6fc1fbb50 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:46.599363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d3994-681d-4e4e-a0d2-a2f6fc1fbb50 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:46.599642 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d3994-681d-4e4e-a0d2-a2f6fc1fbb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:46.628933 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bb4d3994-681d-4e4e-a0d2-a2f6fc1fbb50 None None] MFA Rules not processed for user `b82adabf2646455597fc5802a2ea3819`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:46.630161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bb4d3994-681d-4e4e-a0d2-a2f6fc1fbb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:46.699553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1394/2802] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:46.711746 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bef8e5-1a97-4a0a-886d-0ce945d239c2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:46.713579 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-52bef8e5-1a97-4a0a-886d-0ce945d239c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:46.773363 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bef8e5-1a97-4a0a-886d-0ce945d239c2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:46.777091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa31188-c32b-46d7-a4c3-503c5b7a73ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:46.777757 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa31188-c32b-46d7-a4c3-503c5b7a73ab service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:46.778033 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa31188-c32b-46d7-a4c3-503c5b7a73ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:46.778303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa31188-c32b-46d7-a4c3-503c5b7a73ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:46.788254 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa31188-c32b-46d7-a4c3-503c5b7a73ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:46.789691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa31188-c32b-46d7-a4c3-503c5b7a73ab service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:46.801221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1409/2803] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:47.252840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d3994-681d-4e4e-a0d2-a2f6fc1fbb50 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:47.321928 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d3994-681d-4e4e-a0d2-a2f6fc1fbb50 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:47.326676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:47.327351 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:47.327633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:47.327931 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] PATH_INFO: `/v3/users/8a786c7207204ea78d931db647a1e2eb` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:47.329066 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a786c7207204ea78d931db647a1e2eb)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:47.331179 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:47.378434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:47.381386 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a786c7207204ea78d931db647a1e2eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:47.392356 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a786c7207204ea78d931db647a1e2eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:47.404758 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a786c7207204ea78d931db647a1e2eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:47.405426 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a786c7207204ea78d931db647a1e2eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:47.413316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a786c7207204ea78d931db647a1e2eb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:47.419506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1395/2804] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:47 2022] DELETE /identity/v3/users/8a786c7207204ea78d931db647a1e2eb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:23:47.424585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b3b20-f6e4-493c-9912-cc6d3dea2789 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:47.425094 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b3b20-f6e4-493c-9912-cc6d3dea2789 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:47.425204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b3b20-f6e4-493c-9912-cc6d3dea2789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:47.443898 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-678b3b20-f6e4-493c-9912-cc6d3dea2789 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:47.445119 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-678b3b20-f6e4-493c-9912-cc6d3dea2789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:47.506415 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1410/2805] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:47.516279 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:47.517824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:47.575529 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f38d1-355f-4c3c-a078-46014316e95f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:47.579508 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a940056-a377-4615-88c1-1ab49249411f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:47.580126 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a940056-a377-4615-88c1-1ab49249411f service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:47.580406 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a940056-a377-4615-88c1-1ab49249411f service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:47.580679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a940056-a377-4615-88c1-1ab49249411f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:47.590125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a940056-a377-4615-88c1-1ab49249411f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:47.592213 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a940056-a377-4615-88c1-1ab49249411f service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:47.606491 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1396/2806] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:47.910824 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b3b20-f6e4-493c-9912-cc6d3dea2789 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:47.912326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-678b3b20-f6e4-493c-9912-cc6d3dea2789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:47.933083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b3b20-f6e4-493c-9912-cc6d3dea2789 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:47.941793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:47.942580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:47.942865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:47.943174 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] PATH_INFO: `/v3/projects/7104f49476a840388aecea7a81b1ad10` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:47.944477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7104f49476a840388aecea7a81b1ad10)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:47.946540 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:48.022324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7104f49476a840388aecea7a81b1ad10', 'request_id': 'req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.031346 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7104f49476a840388aecea7a81b1ad10', 'request_id': 'req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.039256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7104f49476a840388aecea7a81b1ad10', 'request_id': 'req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.052948 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] The token cache is being invalidate because project 7104f49476a840388aecea7a81b1ad10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:48.053249 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.053926 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.058603 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1411/2807] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:47 2022] DELETE /identity/v3/projects/7104f49476a840388aecea7a81b1ad10 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:48.059697 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a940056-a377-4615-88c1-1ab49249411f service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:48.069779 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a940056-a377-4615-88c1-1ab49249411f service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:48.139797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285c510-8ea6-4dcd-bcc8-c37055579b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:48.140703 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285c510-8ea6-4dcd-bcc8-c37055579b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:48.142214 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285c510-8ea6-4dcd-bcc8-c37055579b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:48.142214 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285c510-8ea6-4dcd-bcc8-c37055579b71 admin admin] PATH_INFO: `/v3/users/b82adabf2646455597fc5802a2ea3819` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:48.143715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285c510-8ea6-4dcd-bcc8-c37055579b71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b82adabf2646455597fc5802a2ea3819)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:48.146017 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285c510-8ea6-4dcd-bcc8-c37055579b71 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:48.205849 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d285c510-8ea6-4dcd-bcc8-c37055579b71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b82adabf2646455597fc5802a2ea3819'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.214155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d285c510-8ea6-4dcd-bcc8-c37055579b71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b82adabf2646455597fc5802a2ea3819'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.224617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:48.232825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d285c510-8ea6-4dcd-bcc8-c37055579b71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b82adabf2646455597fc5802a2ea3819'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.249874 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d285c510-8ea6-4dcd-bcc8-c37055579b71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b82adabf2646455597fc5802a2ea3819'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.261549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d285c510-8ea6-4dcd-bcc8-c37055579b71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b82adabf2646455597fc5802a2ea3819'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.269506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1397/2808] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:48 2022] DELETE /identity/v3/users/b82adabf2646455597fc5802a2ea3819 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:48.554832 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:48.556446 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:48.566403 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:48.570697 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fa2d5-e604-4079-80f9-9dbe46c3ea93 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:48.647831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36c6df-61da-47be-baa3-62f0e8167602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:48.648810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36c6df-61da-47be-baa3-62f0e8167602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:48.649607 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36c6df-61da-47be-baa3-62f0e8167602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:48.650046 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36c6df-61da-47be-baa3-62f0e8167602 admin admin] PATH_INFO: `/v3/projects/7b0aba874ec5436b89199bb4469ccf88` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:48.652091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36c6df-61da-47be-baa3-62f0e8167602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b0aba874ec5436b89199bb4469ccf88)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:48.654786 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36c6df-61da-47be-baa3-62f0e8167602 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:48.718477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5e36c6df-61da-47be-baa3-62f0e8167602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b0aba874ec5436b89199bb4469ccf88', 'request_id': 'req-5e36c6df-61da-47be-baa3-62f0e8167602'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.727169 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5e36c6df-61da-47be-baa3-62f0e8167602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b0aba874ec5436b89199bb4469ccf88', 'request_id': 'req-5e36c6df-61da-47be-baa3-62f0e8167602'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.734957 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5e36c6df-61da-47be-baa3-62f0e8167602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b0aba874ec5436b89199bb4469ccf88', 'request_id': 'req-5e36c6df-61da-47be-baa3-62f0e8167602'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.746425 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5e36c6df-61da-47be-baa3-62f0e8167602 admin admin] The token cache is being invalidate because project 7b0aba874ec5436b89199bb4469ccf88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:48.746714 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5e36c6df-61da-47be-baa3-62f0e8167602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.747385 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5e36c6df-61da-47be-baa3-62f0e8167602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:48.749129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1412/2809] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:48 2022] DELETE /identity/v3/projects/7b0aba874ec5436b89199bb4469ccf88 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:48.755693 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d86b95-0d67-4eb2-85d3-3c3782dbcbf8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:48.756421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d86b95-0d67-4eb2-85d3-3c3782dbcbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:48.756421 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d86b95-0d67-4eb2-85d3-3c3782dbcbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:48.780774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d6d86b95-0d67-4eb2-85d3-3c3782dbcbf8 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:48.782493 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d6d86b95-0d67-4eb2-85d3-3c3782dbcbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:48.855326 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1398/2810] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:48.858758 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfd0a9-6cfd-45a1-b4b2-b4b963acbe88 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:48.859077 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfd0a9-6cfd-45a1-b4b2-b4b963acbe88 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:48.859349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfd0a9-6cfd-45a1-b4b2-b4b963acbe88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:48.881787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-15dfd0a9-6cfd-45a1-b4b2-b4b963acbe88 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:48.883017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-15dfd0a9-6cfd-45a1-b4b2-b4b963acbe88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:48.892289 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1413/2811] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:23:48.896558 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d86b95-0d67-4eb2-85d3-3c3782dbcbf8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:48.910041 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d86b95-0d67-4eb2-85d3-3c3782dbcbf8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:48.915369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf87fd3-52a0-4162-ae50-90a230a45e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:48.915983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf87fd3-52a0-4162-ae50-90a230a45e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:48.916262 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf87fd3-52a0-4162-ae50-90a230a45e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:48.916536 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf87fd3-52a0-4162-ae50-90a230a45e82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:48.917186 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf87fd3-52a0-4162-ae50-90a230a45e82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:23:48.917496 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf87fd3-52a0-4162-ae50-90a230a45e82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:48.919136 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf87fd3-52a0-4162-ae50-90a230a45e82 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:48.931467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1399/2812] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:23:48.935831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfd0a9-6cfd-45a1-b4b2-b4b963acbe88 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:48.946062 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfd0a9-6cfd-45a1-b4b2-b4b963acbe88 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:48.951038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a34b8-456b-4268-b199-a7bd6af75b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:48.951763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a34b8-456b-4268-b199-a7bd6af75b42 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:48.952046 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a34b8-456b-4268-b199-a7bd6af75b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:48.952323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a34b8-456b-4268-b199-a7bd6af75b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:48.952978 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a34b8-456b-4268-b199-a7bd6af75b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:48.955715 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a34b8-456b-4268-b199-a7bd6af75b42 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:48.984297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1414/2813] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:23:48 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:48.988748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf87fd3-52a0-4162-ae50-90a230a45e82 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:48.999042 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf87fd3-52a0-4162-ae50-90a230a45e82 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:49.077630 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec09618-f7c0-45f7-9c29-4651a9665c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:49.078563 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec09618-f7c0-45f7-9c29-4651a9665c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:49.079517 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec09618-f7c0-45f7-9c29-4651a9665c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:49.079877 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec09618-f7c0-45f7-9c29-4651a9665c40 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:49.080626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec09618-f7c0-45f7-9c29-4651a9665c40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:49.082645 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec09618-f7c0-45f7-9c29-4651a9665c40 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:49.110620 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1400/2814] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:23:48 2022] POST /identity/v3/users => generated 283 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:49.114793 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a34b8-456b-4268-b199-a7bd6af75b42 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:49.126485 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a34b8-456b-4268-b199-a7bd6af75b42 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:49.131406 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bc7b5-7d03-474e-b464-2ed7687ef064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:49.132027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bc7b5-7d03-474e-b464-2ed7687ef064 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:49.132287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bc7b5-7d03-474e-b464-2ed7687ef064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:49.132556 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bc7b5-7d03-474e-b464-2ed7687ef064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:49.133199 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bc7b5-7d03-474e-b464-2ed7687ef064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:49.135086 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bc7b5-7d03-474e-b464-2ed7687ef064 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:49.146026 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1415/2815] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:23:49.150459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec09618-f7c0-45f7-9c29-4651a9665c40 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:49.162896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec09618-f7c0-45f7-9c29-4651a9665c40 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:49.167876 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c3429-22e0-4769-9d26-d7cbecad6a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:49.168500 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c3429-22e0-4769-9d26-d7cbecad6a3f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:49.168782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c3429-22e0-4769-9d26-d7cbecad6a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:49.169061 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c3429-22e0-4769-9d26-d7cbecad6a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:49.170175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c3429-22e0-4769-9d26-d7cbecad6a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:49.171393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c3429-22e0-4769-9d26-d7cbecad6a3f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:49.195462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1401/2816] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:49.197506 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bc7b5-7d03-474e-b464-2ed7687ef064 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:49.209505 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bc7b5-7d03-474e-b464-2ed7687ef064 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:49.215641 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e17fa-acb8-4206-8b0d-b1e041bab476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:49.216614 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e17fa-acb8-4206-8b0d-b1e041bab476 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:49.217038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e17fa-acb8-4206-8b0d-b1e041bab476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:49.217320 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e17fa-acb8-4206-8b0d-b1e041bab476 admin admin] PATH_INFO: `/v3/projects/6c2c0391e68847a389a56661dcd35d8e/users/6e5a9db5170841cf8df061e06ab92f5b/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:49.246978 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e17fa-acb8-4206-8b0d-b1e041bab476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c2c0391e68847a389a56661dcd35d8e, user_id=6e5a9db5170841cf8df061e06ab92f5b, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:49.257465 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e17fa-acb8-4206-8b0d-b1e041bab476 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:49.275146 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1416/2817] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:49 2022] PUT /identity/v3/projects/6c2c0391e68847a389a56661dcd35d8e/users/6e5a9db5170841cf8df061e06ab92f5b/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:49.287295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463fee6-2a96-4037-a5f8-dbb5293cc55b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:49.287711 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463fee6-2a96-4037-a5f8-dbb5293cc55b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:49.287904 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463fee6-2a96-4037-a5f8-dbb5293cc55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:49.340994 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e463fee6-2a96-4037-a5f8-dbb5293cc55b None None] MFA Rules not processed for user `6e5a9db5170841cf8df061e06ab92f5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:49.342460 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e463fee6-2a96-4037-a5f8-dbb5293cc55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:49.393649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:49.427598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1402/2818] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:49.433045 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e17fa-acb8-4206-8b0d-b1e041bab476 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:49.467932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:49.499995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e17fa-acb8-4206-8b0d-b1e041bab476 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:49.506195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb55ed1-9a1d-4881-8785-b8c3bde779f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:49.506810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb55ed1-9a1d-4881-8785-b8c3bde779f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:49.507140 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb55ed1-9a1d-4881-8785-b8c3bde779f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:49.507434 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb55ed1-9a1d-4881-8785-b8c3bde779f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:49.508060 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb55ed1-9a1d-4881-8785-b8c3bde779f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:49.509982 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb55ed1-9a1d-4881-8785-b8c3bde779f3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:49.538457 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1417/2819] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:23:49 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:49.542503 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463fee6-2a96-4037-a5f8-dbb5293cc55b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:49.554454 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463fee6-2a96-4037-a5f8-dbb5293cc55b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:49.620682 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba7ae9-a01c-43fe-92be-fc6f8638c301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:49.621307 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba7ae9-a01c-43fe-92be-fc6f8638c301 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:49.621591 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba7ae9-a01c-43fe-92be-fc6f8638c301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:49.621869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba7ae9-a01c-43fe-92be-fc6f8638c301 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:49.622506 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba7ae9-a01c-43fe-92be-fc6f8638c301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:49.624549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba7ae9-a01c-43fe-92be-fc6f8638c301 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:49.647633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1403/2820] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:23:49 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:23:49.651576 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb55ed1-9a1d-4881-8785-b8c3bde779f3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:49.663233 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb55ed1-9a1d-4881-8785-b8c3bde779f3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:49.667849 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9a403-553c-4627-8ccf-09bfda833c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:49.668468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9a403-553c-4627-8ccf-09bfda833c9b admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:49.668733 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9a403-553c-4627-8ccf-09bfda833c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:49.669000 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9a403-553c-4627-8ccf-09bfda833c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:49.669631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9a403-553c-4627-8ccf-09bfda833c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:49.671250 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9a403-553c-4627-8ccf-09bfda833c9b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:49.682999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1418/2821] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:49.687110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba7ae9-a01c-43fe-92be-fc6f8638c301 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:49.698000 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba7ae9-a01c-43fe-92be-fc6f8638c301 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:49.702805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0250b9-2419-4ee1-8746-3eba2db53e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:49.703530 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0250b9-2419-4ee1-8746-3eba2db53e7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:49.703811 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0250b9-2419-4ee1-8746-3eba2db53e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:49.704067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0250b9-2419-4ee1-8746-3eba2db53e7d admin admin] PATH_INFO: `/v3/projects/a2b92e96d02b4c3b874bcf2215978a36/users/9bb8442b59254c05af7524235c5e386f/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:49.718881 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0250b9-2419-4ee1-8746-3eba2db53e7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b92e96d02b4c3b874bcf2215978a36, user_id=9bb8442b59254c05af7524235c5e386f, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:49.722120 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0250b9-2419-4ee1-8746-3eba2db53e7d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:49.734971 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1404/2822] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:49 2022] PUT /identity/v3/projects/a2b92e96d02b4c3b874bcf2215978a36/users/9bb8442b59254c05af7524235c5e386f/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:49.738735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9a403-553c-4627-8ccf-09bfda833c9b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:49.748978 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9a403-553c-4627-8ccf-09bfda833c9b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:49.807601 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2c54b-33dc-4845-b1e3-7fff89018b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:49.808134 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2c54b-33dc-4845-b1e3-7fff89018b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:49.808412 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2c54b-33dc-4845-b1e3-7fff89018b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:49.808681 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2c54b-33dc-4845-b1e3-7fff89018b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:49.809324 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2c54b-33dc-4845-b1e3-7fff89018b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:49.810919 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2c54b-33dc-4845-b1e3-7fff89018b05 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:49.821553 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1419/2823] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:49.825659 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0250b9-2419-4ee1-8746-3eba2db53e7d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:49.836522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0250b9-2419-4ee1-8746-3eba2db53e7d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:49.841559 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cffc30-8a36-45c9-91db-a8e0eff58880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:49.842370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cffc30-8a36-45c9-91db-a8e0eff58880 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:49.842569 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cffc30-8a36-45c9-91db-a8e0eff58880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:49.842905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cffc30-8a36-45c9-91db-a8e0eff58880 admin admin] PATH_INFO: `/v3/projects/a2b92e96d02b4c3b874bcf2215978a36/users/9bb8442b59254c05af7524235c5e386f/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:49.845396 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cffc30-8a36-45c9-91db-a8e0eff58880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b92e96d02b4c3b874bcf2215978a36, user_id=9bb8442b59254c05af7524235c5e386f, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:49.848111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cffc30-8a36-45c9-91db-a8e0eff58880 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:49.858899 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1405/2824] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:49 2022] PUT /identity/v3/projects/a2b92e96d02b4c3b874bcf2215978a36/users/9bb8442b59254c05af7524235c5e386f/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:49.868976 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480aefd-1abb-4fc6-acb6-fb20152c0c89 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:49.869339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480aefd-1abb-4fc6-acb6-fb20152c0c89 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:49.869616 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480aefd-1abb-4fc6-acb6-fb20152c0c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:49.901761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7480aefd-1abb-4fc6-acb6-fb20152c0c89 None None] MFA Rules not processed for user `9bb8442b59254c05af7524235c5e386f`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:49.903163 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7480aefd-1abb-4fc6-acb6-fb20152c0c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:49.985539 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1420/2825] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:49.997820 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cffc30-8a36-45c9-91db-a8e0eff58880 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:49.999793 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-14cffc30-8a36-45c9-91db-a8e0eff58880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:50.065544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cffc30-8a36-45c9-91db-a8e0eff58880 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:50.069089 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88694555-d79f-4dd9-b50d-9ec06891d7d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:50.069690 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88694555-d79f-4dd9-b50d-9ec06891d7d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:50.069963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88694555-d79f-4dd9-b50d-9ec06891d7d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:50.070227 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88694555-d79f-4dd9-b50d-9ec06891d7d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:50.079332 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88694555-d79f-4dd9-b50d-9ec06891d7d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:50.080851 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88694555-d79f-4dd9-b50d-9ec06891d7d1 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:50.090675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1406/2826] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:50.167243 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480aefd-1abb-4fc6-acb6-fb20152c0c89 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:50.177928 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480aefd-1abb-4fc6-acb6-fb20152c0c89 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:50.181650 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032011a2-1b20-4190-a82c-80f1d345ff9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:50.182256 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032011a2-1b20-4190-a82c-80f1d345ff9c service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:50.182531 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032011a2-1b20-4190-a82c-80f1d345ff9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:50.182803 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032011a2-1b20-4190-a82c-80f1d345ff9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:50.209186 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:50.248149 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032011a2-1b20-4190-a82c-80f1d345ff9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:50.249774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032011a2-1b20-4190-a82c-80f1d345ff9c service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:50.260366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1421/2827] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:52.613923 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88694555-d79f-4dd9-b50d-9ec06891d7d1 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:52.615467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-88694555-d79f-4dd9-b50d-9ec06891d7d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:52.652434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:52.682840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88694555-d79f-4dd9-b50d-9ec06891d7d1 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:52.687405 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:52.688658 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:52.688938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:52.689208 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] PATH_INFO: `/v3/users/3edc8b9046ac4112a838a9c3b39aecf0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:52.690334 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3edc8b9046ac4112a838a9c3b39aecf0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:52.692282 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:52.738137 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3edc8b9046ac4112a838a9c3b39aecf0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:52.745445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3edc8b9046ac4112a838a9c3b39aecf0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:52.758393 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3edc8b9046ac4112a838a9c3b39aecf0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:52.758996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3edc8b9046ac4112a838a9c3b39aecf0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:52.765935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3edc8b9046ac4112a838a9c3b39aecf0'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:52.772407 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1407/2828] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:52 2022] DELETE /identity/v3/users/3edc8b9046ac4112a838a9c3b39aecf0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:52.942466 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032011a2-1b20-4190-a82c-80f1d345ff9c service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:52.955351 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032011a2-1b20-4190-a82c-80f1d345ff9c service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:53.022683 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:53.023487 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:53.023869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:53.024089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] PATH_INFO: `/v3/projects/4cc8da7b799a4bf49ce080e37a36c1d0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:53.025317 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cc8da7b799a4bf49ce080e37a36c1d0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:53.027380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:53.071613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:53.073133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:53.081349 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cc8da7b799a4bf49ce080e37a36c1d0', 'request_id': 'req-5d13fbb9-9439-4ecd-8851-a304ccf217f7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.086948 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd9ca2-b41d-4fd1-a8f6-60c8a444300d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:53.090189 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cc8da7b799a4bf49ce080e37a36c1d0', 'request_id': 'req-5d13fbb9-9439-4ecd-8851-a304ccf217f7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.097901 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cc8da7b799a4bf49ce080e37a36c1d0', 'request_id': 'req-5d13fbb9-9439-4ecd-8851-a304ccf217f7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.109347 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] The token cache is being invalidate because project 4cc8da7b799a4bf49ce080e37a36c1d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:53.109639 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.110813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.112532 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1422/2829] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:52 2022] DELETE /identity/v3/projects/4cc8da7b799a4bf49ce080e37a36c1d0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:53.116187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:53.117572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:53.147183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba95315-e2fd-4b3b-be68-4c7983e3a3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:53.147824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95315-e2fd-4b3b-be68-4c7983e3a3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:53.148104 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95315-e2fd-4b3b-be68-4c7983e3a3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:53.148376 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95315-e2fd-4b3b-be68-4c7983e3a3e9 admin admin] PATH_INFO: `/v3/users/6e5a9db5170841cf8df061e06ab92f5b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:53.149590 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba95315-e2fd-4b3b-be68-4c7983e3a3e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e5a9db5170841cf8df061e06ab92f5b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:53.151582 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba95315-e2fd-4b3b-be68-4c7983e3a3e9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:53.180328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:53.185598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13fbb9-9439-4ecd-8851-a304ccf217f7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:53.191489 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:53.192366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:53.192851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:53.193410 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] PATH_INFO: `/v3/users/cb2d34d0ab1c4c38b1f55c13f8af08d6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:53.194556 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb2d34d0ab1c4c38b1f55c13f8af08d6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:53.196491 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:53.201080 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4ba95315-e2fd-4b3b-be68-4c7983e3a3e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e5a9db5170841cf8df061e06ab92f5b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.209239 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4ba95315-e2fd-4b3b-be68-4c7983e3a3e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e5a9db5170841cf8df061e06ab92f5b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.210725 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:53.225482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4ba95315-e2fd-4b3b-be68-4c7983e3a3e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e5a9db5170841cf8df061e06ab92f5b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.226198 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4ba95315-e2fd-4b3b-be68-4c7983e3a3e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e5a9db5170841cf8df061e06ab92f5b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.234034 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4ba95315-e2fd-4b3b-be68-4c7983e3a3e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e5a9db5170841cf8df061e06ab92f5b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.240755 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1408/2830] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:53 2022] DELETE /identity/v3/users/6e5a9db5170841cf8df061e06ab92f5b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:23:53.245155 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326300f-94e3-4733-be93-59b695164990 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:53.245383 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326300f-94e3-4733-be93-59b695164990 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:53.245663 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326300f-94e3-4733-be93-59b695164990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:53.246572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb2d34d0ab1c4c38b1f55c13f8af08d6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.256368 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb2d34d0ab1c4c38b1f55c13f8af08d6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.257083 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb2d34d0ab1c4c38b1f55c13f8af08d6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.266011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a326300f-94e3-4733-be93-59b695164990 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:53.267271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a326300f-94e3-4733-be93-59b695164990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:53.269811 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb2d34d0ab1c4c38b1f55c13f8af08d6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.277080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb2d34d0ab1c4c38b1f55c13f8af08d6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.283385 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1423/2831] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:53 2022] DELETE /identity/v3/users/cb2d34d0ab1c4c38b1f55c13f8af08d6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:23:53.331652 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1409/2832] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:23:53.341161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:53.343241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:53.404303 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172cb02-51a1-4e2b-89bf-b7674a761109 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:53.408747 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fad48-e089-4db5-aa5d-d3bddce2a21c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:53.409341 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fad48-e089-4db5-aa5d-d3bddce2a21c service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:53.409618 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fad48-e089-4db5-aa5d-d3bddce2a21c service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:53.409933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fad48-e089-4db5-aa5d-d3bddce2a21c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:53.460638 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326300f-94e3-4733-be93-59b695164990 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:53.462196 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a326300f-94e3-4733-be93-59b695164990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:53.475946 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fad48-e089-4db5-aa5d-d3bddce2a21c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:53.477390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fad48-e089-4db5-aa5d-d3bddce2a21c service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:53.481244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326300f-94e3-4733-be93-59b695164990 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:53.488371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1424/2833] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:53.493260 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:53.493917 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:53.494270 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:53.494551 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] PATH_INFO: `/v3/projects/a89d432a742849419bc76f45b2c306c5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:53.495805 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89d432a742849419bc76f45b2c306c5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:53.497637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:53.550114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89d432a742849419bc76f45b2c306c5', 'request_id': 'req-5533d593-b7b1-4ccc-a197-11d2f6348a2d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.558480 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89d432a742849419bc76f45b2c306c5', 'request_id': 'req-5533d593-b7b1-4ccc-a197-11d2f6348a2d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.566831 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89d432a742849419bc76f45b2c306c5', 'request_id': 'req-5533d593-b7b1-4ccc-a197-11d2f6348a2d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.581337 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] The token cache is being invalidate because project a89d432a742849419bc76f45b2c306c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:53.581626 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.582401 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.583943 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1410/2834] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:53 2022] DELETE /identity/v3/projects/a89d432a742849419bc76f45b2c306c5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:23:53.785873 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fad48-e089-4db5-aa5d-d3bddce2a21c service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:53.787344 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-408fad48-e089-4db5-aa5d-d3bddce2a21c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:53.809995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fad48-e089-4db5-aa5d-d3bddce2a21c service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:53.837739 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:53.881954 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:53.882598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:53.882882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:53.883202 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] PATH_INFO: `/v3/projects/6c2c0391e68847a389a56661dcd35d8e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:53.884366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c2c0391e68847a389a56661dcd35d8e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:53.886283 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:53.936390 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c2c0391e68847a389a56661dcd35d8e', 'request_id': 'req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.946004 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c2c0391e68847a389a56661dcd35d8e', 'request_id': 'req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.952338 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c2c0391e68847a389a56661dcd35d8e', 'request_id': 'req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.964801 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] The token cache is being invalidate because project 6c2c0391e68847a389a56661dcd35d8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:53.965185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.965924 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:53.967448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1425/2835] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:53 2022] DELETE /identity/v3/projects/6c2c0391e68847a389a56661dcd35d8e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:53.971379 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:53.972861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:53.986691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533d593-b7b1-4ccc-a197-11d2f6348a2d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:54.037218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:54.045467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47997670-77a3-441d-9b11-f7ed8e8bd870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:54.046257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47997670-77a3-441d-9b11-f7ed8e8bd870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:54.046742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47997670-77a3-441d-9b11-f7ed8e8bd870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:54.047159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47997670-77a3-441d-9b11-f7ed8e8bd870 admin admin] PATH_INFO: `/v3/users/9bb8442b59254c05af7524235c5e386f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:54.049150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47997670-77a3-441d-9b11-f7ed8e8bd870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb8442b59254c05af7524235c5e386f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:54.051247 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47997670-77a3-441d-9b11-f7ed8e8bd870 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:54.100952 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-47997670-77a3-441d-9b11-f7ed8e8bd870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb8442b59254c05af7524235c5e386f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:54.108251 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-47997670-77a3-441d-9b11-f7ed8e8bd870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb8442b59254c05af7524235c5e386f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:54.120914 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-47997670-77a3-441d-9b11-f7ed8e8bd870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb8442b59254c05af7524235c5e386f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:54.121553 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-47997670-77a3-441d-9b11-f7ed8e8bd870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb8442b59254c05af7524235c5e386f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:54.129259 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-47997670-77a3-441d-9b11-f7ed8e8bd870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb8442b59254c05af7524235c5e386f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:54.135093 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1411/2836] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:53 2022] DELETE /identity/v3/users/9bb8442b59254c05af7524235c5e386f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:54.403538 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:54.405051 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:54.418137 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbacf4-88e4-4d4d-af5a-4d9542bf0b1c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:54.477266 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6aa36-31ad-425a-9275-87ec1992d316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:54.477922 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6aa36-31ad-425a-9275-87ec1992d316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:54.478201 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6aa36-31ad-425a-9275-87ec1992d316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:54.478474 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6aa36-31ad-425a-9275-87ec1992d316 admin admin] PATH_INFO: `/v3/projects/a2b92e96d02b4c3b874bcf2215978a36` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:54.479624 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6aa36-31ad-425a-9275-87ec1992d316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b92e96d02b4c3b874bcf2215978a36)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:54.481518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6aa36-31ad-425a-9275-87ec1992d316 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:54.527695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d7a6aa36-31ad-425a-9275-87ec1992d316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b92e96d02b4c3b874bcf2215978a36', 'request_id': 'req-d7a6aa36-31ad-425a-9275-87ec1992d316'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:54.534666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d7a6aa36-31ad-425a-9275-87ec1992d316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b92e96d02b4c3b874bcf2215978a36', 'request_id': 'req-d7a6aa36-31ad-425a-9275-87ec1992d316'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:54.535930 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:54.541537 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d7a6aa36-31ad-425a-9275-87ec1992d316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b92e96d02b4c3b874bcf2215978a36', 'request_id': 'req-d7a6aa36-31ad-425a-9275-87ec1992d316'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:54.554211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d7a6aa36-31ad-425a-9275-87ec1992d316 admin admin] The token cache is being invalidate because project a2b92e96d02b4c3b874bcf2215978a36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:54.554496 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d7a6aa36-31ad-425a-9275-87ec1992d316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:54.555217 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d7a6aa36-31ad-425a-9275-87ec1992d316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:54.556764 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1426/2837] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:54 2022] DELETE /identity/v3/projects/a2b92e96d02b4c3b874bcf2215978a36 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:54.561713 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402617dc-ee19-40e5-a24d-fcc8ca608452 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:54.561938 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402617dc-ee19-40e5-a24d-fcc8ca608452 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:54.562211 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402617dc-ee19-40e5-a24d-fcc8ca608452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:54.582036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-402617dc-ee19-40e5-a24d-fcc8ca608452 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:54.583275 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-402617dc-ee19-40e5-a24d-fcc8ca608452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:54.644586 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1412/2838] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:54.652550 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a476db4-9e6c-4bdc-90a0-1cc69c2e2815 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:54.652830 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a476db4-9e6c-4bdc-90a0-1cc69c2e2815 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:54.653101 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a476db4-9e6c-4bdc-90a0-1cc69c2e2815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:54.673703 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3a476db4-9e6c-4bdc-90a0-1cc69c2e2815 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:54.674903 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3a476db4-9e6c-4bdc-90a0-1cc69c2e2815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:54.682687 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1427/2839] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:54.686808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402617dc-ee19-40e5-a24d-fcc8ca608452 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:54.696987 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402617dc-ee19-40e5-a24d-fcc8ca608452 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:54.701437 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43600fc9-2599-4e88-a86c-e63b37895b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:54.702027 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43600fc9-2599-4e88-a86c-e63b37895b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:54.702303 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43600fc9-2599-4e88-a86c-e63b37895b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:54.702571 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43600fc9-2599-4e88-a86c-e63b37895b4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:54.703238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43600fc9-2599-4e88-a86c-e63b37895b4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:23:54.703597 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43600fc9-2599-4e88-a86c-e63b37895b4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:54.705283 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43600fc9-2599-4e88-a86c-e63b37895b4e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:54.716649 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1413/2840] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:23:54.721006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a476db4-9e6c-4bdc-90a0-1cc69c2e2815 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:54.730056 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a476db4-9e6c-4bdc-90a0-1cc69c2e2815 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:54.734669 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663a266-55b1-4824-a1d7-0615b920c4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:54.736154 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a266-55b1-4824-a1d7-0615b920c4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:54.736436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a266-55b1-4824-a1d7-0615b920c4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:54.736708 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a266-55b1-4824-a1d7-0615b920c4b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:54.737328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663a266-55b1-4824-a1d7-0615b920c4b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:54.739307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663a266-55b1-4824-a1d7-0615b920c4b0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:54.772840 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1428/2841] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:23:54 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:54.777015 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43600fc9-2599-4e88-a86c-e63b37895b4e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:54.786953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43600fc9-2599-4e88-a86c-e63b37895b4e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:54.842684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208922c4-7ac1-4751-a833-b6d85a197566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:54.843257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208922c4-7ac1-4751-a833-b6d85a197566 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:54.843527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208922c4-7ac1-4751-a833-b6d85a197566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:54.843797 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208922c4-7ac1-4751-a833-b6d85a197566 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:54.844453 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208922c4-7ac1-4751-a833-b6d85a197566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:54.846400 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208922c4-7ac1-4751-a833-b6d85a197566 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:54.855165 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:54.869794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1414/2842] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:23:54 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:54.874161 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663a266-55b1-4824-a1d7-0615b920c4b0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:54.885267 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663a266-55b1-4824-a1d7-0615b920c4b0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:54.889890 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd4da3-b866-4712-b97c-a32639c3c816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:54.890486 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd4da3-b866-4712-b97c-a32639c3c816 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:54.890766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd4da3-b866-4712-b97c-a32639c3c816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:54.891240 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd4da3-b866-4712-b97c-a32639c3c816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:54.891869 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd4da3-b866-4712-b97c-a32639c3c816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:54.893448 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd4da3-b866-4712-b97c-a32639c3c816 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:54.903339 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1429/2843] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:23:54.907772 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208922c4-7ac1-4751-a833-b6d85a197566 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:54.917620 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208922c4-7ac1-4751-a833-b6d85a197566 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:54.922229 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56097d1d-35ae-4f5b-8967-59f2f0f7cc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:54.923478 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56097d1d-35ae-4f5b-8967-59f2f0f7cc8a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:54.923753 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56097d1d-35ae-4f5b-8967-59f2f0f7cc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:54.924025 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56097d1d-35ae-4f5b-8967-59f2f0f7cc8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:54.924685 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56097d1d-35ae-4f5b-8967-59f2f0f7cc8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:54.926258 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56097d1d-35ae-4f5b-8967-59f2f0f7cc8a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:54.936614 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1415/2844] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:54.940665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd4da3-b866-4712-b97c-a32639c3c816 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:54.950615 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd4da3-b866-4712-b97c-a32639c3c816 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:54.955694 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67693e05-a0b7-43fe-b075-236190529574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:54.956366 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67693e05-a0b7-43fe-b075-236190529574 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:54.956647 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67693e05-a0b7-43fe-b075-236190529574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:54.956923 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67693e05-a0b7-43fe-b075-236190529574 admin admin] PATH_INFO: `/v3/projects/54159ca1121d4a7fbc1e5b34dcf09dd8/users/5cbedb7a131745df8f814077198c815a/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:54.972171 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67693e05-a0b7-43fe-b075-236190529574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54159ca1121d4a7fbc1e5b34dcf09dd8, user_id=5cbedb7a131745df8f814077198c815a, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:54.974823 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67693e05-a0b7-43fe-b075-236190529574 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:54.988335 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1430/2845] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:54 2022] PUT /identity/v3/projects/54159ca1121d4a7fbc1e5b34dcf09dd8/users/5cbedb7a131745df8f814077198c815a/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:54.999743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d7480-2f0c-48d2-953e-d0306adce8f5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:55.000271 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d7480-2f0c-48d2-953e-d0306adce8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:55.000771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d7480-2f0c-48d2-953e-d0306adce8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:55.027930 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-295d7480-2f0c-48d2-953e-d0306adce8f5 None None] MFA Rules not processed for user `5cbedb7a131745df8f814077198c815a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:55.029154 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-295d7480-2f0c-48d2-953e-d0306adce8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:55.092007 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1416/2846] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:55.097239 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67693e05-a0b7-43fe-b075-236190529574 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:55.159070 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67693e05-a0b7-43fe-b075-236190529574 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:55.163725 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cdb43-7669-49b0-bfaf-1dc6cf23a33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:55.164317 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cdb43-7669-49b0-bfaf-1dc6cf23a33f admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:55.164593 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cdb43-7669-49b0-bfaf-1dc6cf23a33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:55.164862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cdb43-7669-49b0-bfaf-1dc6cf23a33f admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:55.165490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643cdb43-7669-49b0-bfaf-1dc6cf23a33f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:55.167408 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643cdb43-7669-49b0-bfaf-1dc6cf23a33f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:55.194022 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1431/2847] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:23:55 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:55.198011 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d7480-2f0c-48d2-953e-d0306adce8f5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:55.208445 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d7480-2f0c-48d2-953e-d0306adce8f5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:55.264355 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2256de-8355-4254-ab2e-3c122ce3fe1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:55.264965 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2256de-8355-4254-ab2e-3c122ce3fe1b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:55.265244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2256de-8355-4254-ab2e-3c122ce3fe1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:55.265512 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2256de-8355-4254-ab2e-3c122ce3fe1b admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:55.266708 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2256de-8355-4254-ab2e-3c122ce3fe1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:55.268672 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2256de-8355-4254-ab2e-3c122ce3fe1b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:55.288727 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1417/2848] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:23:55 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:55.292676 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cdb43-7669-49b0-bfaf-1dc6cf23a33f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:55.303082 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cdb43-7669-49b0-bfaf-1dc6cf23a33f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:55.307666 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6643008-bdca-4cfa-9892-2fd953cf7751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:55.308260 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6643008-bdca-4cfa-9892-2fd953cf7751 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:55.308535 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6643008-bdca-4cfa-9892-2fd953cf7751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:55.308808 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6643008-bdca-4cfa-9892-2fd953cf7751 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:55.309437 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6643008-bdca-4cfa-9892-2fd953cf7751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:55.311015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6643008-bdca-4cfa-9892-2fd953cf7751 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:55.321452 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1432/2849] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:55.325446 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2256de-8355-4254-ab2e-3c122ce3fe1b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:55.335721 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2256de-8355-4254-ab2e-3c122ce3fe1b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:55.340266 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e4883-76b9-4678-ae82-3d39aa1f3662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:55.340903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e4883-76b9-4678-ae82-3d39aa1f3662 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:55.341181 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e4883-76b9-4678-ae82-3d39aa1f3662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:55.341456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e4883-76b9-4678-ae82-3d39aa1f3662 admin admin] PATH_INFO: `/v3/projects/43336826602b499fa3d8304758eb51fb/users/0a0326500abd433687e3c8b439a2f5c1/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:55.356547 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e4883-76b9-4678-ae82-3d39aa1f3662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43336826602b499fa3d8304758eb51fb, user_id=0a0326500abd433687e3c8b439a2f5c1, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:55.359152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e4883-76b9-4678-ae82-3d39aa1f3662 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:55.371860 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1418/2850] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:55 2022] PUT /identity/v3/projects/43336826602b499fa3d8304758eb51fb/users/0a0326500abd433687e3c8b439a2f5c1/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:23:55.376527 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6643008-bdca-4cfa-9892-2fd953cf7751 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:55.386285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6643008-bdca-4cfa-9892-2fd953cf7751 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:55.435842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:55.443665 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f15e59-e09b-4d54-add6-a60b82188aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:55.444433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f15e59-e09b-4d54-add6-a60b82188aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:55.444916 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f15e59-e09b-4d54-add6-a60b82188aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:55.445241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f15e59-e09b-4d54-add6-a60b82188aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:55.446050 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f15e59-e09b-4d54-add6-a60b82188aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:55.447735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f15e59-e09b-4d54-add6-a60b82188aa0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:55.458397 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1433/2851] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:55.462403 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e4883-76b9-4678-ae82-3d39aa1f3662 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:55.473402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e4883-76b9-4678-ae82-3d39aa1f3662 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:55.478203 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd55196-4921-4620-8c30-498684e7610a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:55.478855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd55196-4921-4620-8c30-498684e7610a admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:55.479201 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd55196-4921-4620-8c30-498684e7610a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:55.479481 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd55196-4921-4620-8c30-498684e7610a admin admin] PATH_INFO: `/v3/projects/43336826602b499fa3d8304758eb51fb/users/0a0326500abd433687e3c8b439a2f5c1/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:55.481469 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd55196-4921-4620-8c30-498684e7610a admin admin] RBAC: Authorizing `identity:create_grant(project_id=43336826602b499fa3d8304758eb51fb, user_id=0a0326500abd433687e3c8b439a2f5c1, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:55.484139 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd55196-4921-4620-8c30-498684e7610a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:55.494363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1419/2852] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:55 2022] PUT /identity/v3/projects/43336826602b499fa3d8304758eb51fb/users/0a0326500abd433687e3c8b439a2f5c1/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:55.504834 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07e1d3-5996-4831-8a5d-b451fae30b3f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:55.505124 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07e1d3-5996-4831-8a5d-b451fae30b3f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:55.505398 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07e1d3-5996-4831-8a5d-b451fae30b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:55.535173 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-df07e1d3-5996-4831-8a5d-b451fae30b3f None None] MFA Rules not processed for user `0a0326500abd433687e3c8b439a2f5c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:55.536383 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-df07e1d3-5996-4831-8a5d-b451fae30b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:55.603963 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1434/2853] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:55.615450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd55196-4921-4620-8c30-498684e7610a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:55.617551 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7bd55196-4921-4620-8c30-498684e7610a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:55.647389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:55.681858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd55196-4921-4620-8c30-498684e7610a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:55.685587 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155027aa-d077-4a34-bd0b-93bf8e7c67fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:55.686196 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155027aa-d077-4a34-bd0b-93bf8e7c67fc service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:55.686472 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155027aa-d077-4a34-bd0b-93bf8e7c67fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:55.686742 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155027aa-d077-4a34-bd0b-93bf8e7c67fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:55.695882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155027aa-d077-4a34-bd0b-93bf8e7c67fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:55.697319 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155027aa-d077-4a34-bd0b-93bf8e7c67fc service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:55.706803 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1420/2854] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:56.076800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07e1d3-5996-4831-8a5d-b451fae30b3f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:56.138241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07e1d3-5996-4831-8a5d-b451fae30b3f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:56.143286 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:56.143906 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:56.144185 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:56.144455 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] PATH_INFO: `/v3/users/5cbedb7a131745df8f814077198c815a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:56.145565 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cbedb7a131745df8f814077198c815a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:56.147585 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:56.193254 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cbedb7a131745df8f814077198c815a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.200572 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cbedb7a131745df8f814077198c815a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.201206 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cbedb7a131745df8f814077198c815a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.213084 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cbedb7a131745df8f814077198c815a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.218317 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:56.222006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cbedb7a131745df8f814077198c815a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.229422 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1435/2855] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:56 2022] DELETE /identity/v3/users/5cbedb7a131745df8f814077198c815a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:56.233887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226d5b6-46ba-4aee-86f8-507bd019fe29 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:56.234370 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226d5b6-46ba-4aee-86f8-507bd019fe29 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:56.234488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226d5b6-46ba-4aee-86f8-507bd019fe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:56.254358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c226d5b6-46ba-4aee-86f8-507bd019fe29 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:56.255564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c226d5b6-46ba-4aee-86f8-507bd019fe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:56.313430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1421/2856] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:23:56.323114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:56.324531 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:56.380840 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1643d80-23ba-4477-b5c4-06af0b279d3e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:56.384542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca320e-eab8-407d-a855-bfc429b962af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:56.385136 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca320e-eab8-407d-a855-bfc429b962af service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:56.385416 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca320e-eab8-407d-a855-bfc429b962af service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:56.385689 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca320e-eab8-407d-a855-bfc429b962af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:56.394952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca320e-eab8-407d-a855-bfc429b962af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:56.396348 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca320e-eab8-407d-a855-bfc429b962af service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:56.405838 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1436/2857] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:56.674825 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226d5b6-46ba-4aee-86f8-507bd019fe29 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:56.676381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c226d5b6-46ba-4aee-86f8-507bd019fe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:56.692564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226d5b6-46ba-4aee-86f8-507bd019fe29 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:56.697381 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:56.698013 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:56.698292 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:56.698681 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] PATH_INFO: `/v3/projects/54159ca1121d4a7fbc1e5b34dcf09dd8` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:56.699684 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54159ca1121d4a7fbc1e5b34dcf09dd8)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:56.701588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:56.757562 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54159ca1121d4a7fbc1e5b34dcf09dd8', 'request_id': 'req-a656e2b3-0e0b-45be-b10f-80a6cdce6569'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.764288 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54159ca1121d4a7fbc1e5b34dcf09dd8', 'request_id': 'req-a656e2b3-0e0b-45be-b10f-80a6cdce6569'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.770675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54159ca1121d4a7fbc1e5b34dcf09dd8', 'request_id': 'req-a656e2b3-0e0b-45be-b10f-80a6cdce6569'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.782963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] The token cache is being invalidate because project 54159ca1121d4a7fbc1e5b34dcf09dd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:56.783278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.783893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.785374 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1422/2858] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:56 2022] DELETE /identity/v3/projects/54159ca1121d4a7fbc1e5b34dcf09dd8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:56.789497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca320e-eab8-407d-a855-bfc429b962af service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:56.799526 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca320e-eab8-407d-a855-bfc429b962af service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:56.854089 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65b10c-0ef2-4db3-bf2f-94a73448388d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:56.854705 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b10c-0ef2-4db3-bf2f-94a73448388d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:56.854983 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b10c-0ef2-4db3-bf2f-94a73448388d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:56.855287 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b10c-0ef2-4db3-bf2f-94a73448388d admin admin] PATH_INFO: `/v3/users/0a0326500abd433687e3c8b439a2f5c1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:56.856401 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65b10c-0ef2-4db3-bf2f-94a73448388d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a0326500abd433687e3c8b439a2f5c1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:56.858307 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65b10c-0ef2-4db3-bf2f-94a73448388d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:56.902580 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd65b10c-0ef2-4db3-bf2f-94a73448388d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a0326500abd433687e3c8b439a2f5c1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.910528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd65b10c-0ef2-4db3-bf2f-94a73448388d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a0326500abd433687e3c8b439a2f5c1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.911343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd65b10c-0ef2-4db3-bf2f-94a73448388d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a0326500abd433687e3c8b439a2f5c1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.924114 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd65b10c-0ef2-4db3-bf2f-94a73448388d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a0326500abd433687e3c8b439a2f5c1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.930960 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd65b10c-0ef2-4db3-bf2f-94a73448388d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a0326500abd433687e3c8b439a2f5c1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:56.936797 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1437/2859] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:56 2022] DELETE /identity/v3/users/0a0326500abd433687e3c8b439a2f5c1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:57.212096 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:57.213747 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:57.232482 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656e2b3-0e0b-45be-b10f-80a6cdce6569 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:57.279554 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:57.293046 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:57.293676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:57.293951 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:57.294222 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085 admin admin] PATH_INFO: `/v3/projects/43336826602b499fa3d8304758eb51fb` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:57.295369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43336826602b499fa3d8304758eb51fb)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:57.297264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:57.342827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43336826602b499fa3d8304758eb51fb', 'request_id': 'req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:57.348366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43336826602b499fa3d8304758eb51fb', 'request_id': 'req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:57.354434 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43336826602b499fa3d8304758eb51fb', 'request_id': 'req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:57.366497 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085 admin admin] The token cache is being invalidate because project 43336826602b499fa3d8304758eb51fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:57.366785 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:57.367414 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8a9b8f22-ac74-4214-9de3-0bbfdbcf2085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:57.368864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1423/2860] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:57 2022] DELETE /identity/v3/projects/43336826602b499fa3d8304758eb51fb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:57.373745 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f290938-1723-4187-8acd-33d7416c186e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:57.374203 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f290938-1723-4187-8acd-33d7416c186e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:57.374354 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f290938-1723-4187-8acd-33d7416c186e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:57.394844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3f290938-1723-4187-8acd-33d7416c186e None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:57.396097 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3f290938-1723-4187-8acd-33d7416c186e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:57.408017 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:57.456546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1438/2861] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:57.519416 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794420d4-da90-47ef-943c-c1da1f49a9ac None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:57.519697 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794420d4-da90-47ef-943c-c1da1f49a9ac None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:57.519997 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794420d4-da90-47ef-943c-c1da1f49a9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:57.539945 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-794420d4-da90-47ef-943c-c1da1f49a9ac None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:57.541145 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-794420d4-da90-47ef-943c-c1da1f49a9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:57.549706 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1424/2862] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:57.554106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f290938-1723-4187-8acd-33d7416c186e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:57.564498 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f290938-1723-4187-8acd-33d7416c186e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:57.569080 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6c326-6e54-445c-8bca-17522da0c062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:57.569682 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6c326-6e54-445c-8bca-17522da0c062 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:57.569962 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6c326-6e54-445c-8bca-17522da0c062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:57.570235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6c326-6e54-445c-8bca-17522da0c062 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:57.570882 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6c326-6e54-445c-8bca-17522da0c062 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:23:57.571221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6c326-6e54-445c-8bca-17522da0c062 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:57.572810 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6c326-6e54-445c-8bca-17522da0c062 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:57.584668 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1439/2863] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:23:57.588927 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794420d4-da90-47ef-943c-c1da1f49a9ac None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:57.598110 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794420d4-da90-47ef-943c-c1da1f49a9ac None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:57.602675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ef9e5-962b-4cf0-a8a6-45fd17becaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:57.603302 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ef9e5-962b-4cf0-a8a6-45fd17becaf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:57.603861 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ef9e5-962b-4cf0-a8a6-45fd17becaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:57.604126 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ef9e5-962b-4cf0-a8a6-45fd17becaf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:57.604760 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ef9e5-962b-4cf0-a8a6-45fd17becaf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:57.606941 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ef9e5-962b-4cf0-a8a6-45fd17becaf0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:57.635456 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1425/2864] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:23:57 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:23:57.639805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6c326-6e54-445c-8bca-17522da0c062 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:57.650061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6c326-6e54-445c-8bca-17522da0c062 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:57.708090 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbaef69-1d75-4944-8cc5-058dd77ca92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:57.708631 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaef69-1d75-4944-8cc5-058dd77ca92b admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:57.708911 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaef69-1d75-4944-8cc5-058dd77ca92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:57.709181 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaef69-1d75-4944-8cc5-058dd77ca92b admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:57.709814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbaef69-1d75-4944-8cc5-058dd77ca92b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:57.711768 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbaef69-1d75-4944-8cc5-058dd77ca92b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:57.732709 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1440/2865] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:23:57 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:23:57.739754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ef9e5-962b-4cf0-a8a6-45fd17becaf0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:57.751527 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ef9e5-962b-4cf0-a8a6-45fd17becaf0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:57.756230 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ca9e4-8bdb-41a0-bf5d-978c50bdf833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:57.756912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ca9e4-8bdb-41a0-bf5d-978c50bdf833 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:57.757254 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ca9e4-8bdb-41a0-bf5d-978c50bdf833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:57.757594 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ca9e4-8bdb-41a0-bf5d-978c50bdf833 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:57.758324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ca9e4-8bdb-41a0-bf5d-978c50bdf833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:57.760014 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ca9e4-8bdb-41a0-bf5d-978c50bdf833 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:57.770312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1426/2866] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:57.774379 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbaef69-1d75-4944-8cc5-058dd77ca92b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:57.784630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbaef69-1d75-4944-8cc5-058dd77ca92b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:57.789180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5b898-5603-4dc0-a641-2183044c1611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:57.789774 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5b898-5603-4dc0-a641-2183044c1611 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:57.790052 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5b898-5603-4dc0-a641-2183044c1611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:57.790328 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5b898-5603-4dc0-a641-2183044c1611 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:57.790965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5b898-5603-4dc0-a641-2183044c1611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:57.792575 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5b898-5603-4dc0-a641-2183044c1611 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:57.803824 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1441/2867] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:57.807746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ca9e4-8bdb-41a0-bf5d-978c50bdf833 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:57.818366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ca9e4-8bdb-41a0-bf5d-978c50bdf833 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:57.822997 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588c847-575a-4b07-9eb2-0ba4a90bb5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:57.823675 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588c847-575a-4b07-9eb2-0ba4a90bb5f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:57.823952 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588c847-575a-4b07-9eb2-0ba4a90bb5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:57.824224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588c847-575a-4b07-9eb2-0ba4a90bb5f9 admin admin] PATH_INFO: `/v3/projects/878e4994ef914c2ba57e810d6e875a86/users/f5757557df4845378463188c93c7279f/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:57.838754 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588c847-575a-4b07-9eb2-0ba4a90bb5f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=878e4994ef914c2ba57e810d6e875a86, user_id=f5757557df4845378463188c93c7279f, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:57.841440 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588c847-575a-4b07-9eb2-0ba4a90bb5f9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:57.856209 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1427/2868] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:57 2022] PUT /identity/v3/projects/878e4994ef914c2ba57e810d6e875a86/users/f5757557df4845378463188c93c7279f/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:57.867167 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c8ce1-5b8d-48df-922f-97fdbb46b92d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:57.867501 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c8ce1-5b8d-48df-922f-97fdbb46b92d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:57.867796 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c8ce1-5b8d-48df-922f-97fdbb46b92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:57.896573 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b83c8ce1-5b8d-48df-922f-97fdbb46b92d None None] MFA Rules not processed for user `f5757557df4845378463188c93c7279f`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:57.897782 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b83c8ce1-5b8d-48df-922f-97fdbb46b92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:57.962180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1442/2869] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:57.973750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588c847-575a-4b07-9eb2-0ba4a90bb5f9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:57.975467 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0588c847-575a-4b07-9eb2-0ba4a90bb5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:58.034422 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588c847-575a-4b07-9eb2-0ba4a90bb5f9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:58.038285 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe1ba3-df7f-43bc-936a-0914dbb384f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:58.038887 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1ba3-df7f-43bc-936a-0914dbb384f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:58.039280 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1ba3-df7f-43bc-936a-0914dbb384f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:58.039607 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1ba3-df7f-43bc-936a-0914dbb384f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:58.049128 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe1ba3-df7f-43bc-936a-0914dbb384f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:58.050549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe1ba3-df7f-43bc-936a-0914dbb384f2 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:58.059205 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1428/2870] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:58.082178 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c8ce1-5b8d-48df-922f-97fdbb46b92d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:58.095851 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:58.145353 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c8ce1-5b8d-48df-922f-97fdbb46b92d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:58.149932 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:58.150552 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:58.150831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:58.151131 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] PATH_INFO: `/v3/users/f5757557df4845378463188c93c7279f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:58.152299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5757557df4845378463188c93c7279f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:58.154200 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:58.199517 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5757557df4845378463188c93c7279f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:58.206951 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5757557df4845378463188c93c7279f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:58.207682 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5757557df4845378463188c93c7279f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:58.220468 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5757557df4845378463188c93c7279f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:58.227245 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5757557df4845378463188c93c7279f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:58.233110 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1443/2871] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:23:58 2022] DELETE /identity/v3/users/f5757557df4845378463188c93c7279f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:23:58.238304 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228527d-5adb-4adf-ae31-d92fb80f3cc9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:58.238591 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228527d-5adb-4adf-ae31-d92fb80f3cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:58.238896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228527d-5adb-4adf-ae31-d92fb80f3cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:58.258450 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1228527d-5adb-4adf-ae31-d92fb80f3cc9 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:58.259698 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1228527d-5adb-4adf-ae31-d92fb80f3cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:58.272062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:58.321278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1429/2872] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:58.331780 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:58.333175 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:58.389488 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d9a8c-7fd7-4a74-88aa-0c2321743d28 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:58.393805 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d04ab-4b8f-4426-bcdf-74c68610fc9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:58.394399 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d04ab-4b8f-4426-bcdf-74c68610fc9a service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:58.394679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d04ab-4b8f-4426-bcdf-74c68610fc9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:58.394952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d04ab-4b8f-4426-bcdf-74c68610fc9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:58.403866 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d04ab-4b8f-4426-bcdf-74c68610fc9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:58.405282 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d04ab-4b8f-4426-bcdf-74c68610fc9a service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:58.415299 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1444/2873] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:23:58.683834 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228527d-5adb-4adf-ae31-d92fb80f3cc9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:58.685257 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1228527d-5adb-4adf-ae31-d92fb80f3cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:58.700505 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228527d-5adb-4adf-ae31-d92fb80f3cc9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:58.707906 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a54c1-4669-4aca-8265-b957775f00f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:58.708540 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a54c1-4669-4aca-8265-b957775f00f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:58.708808 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a54c1-4669-4aca-8265-b957775f00f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:58.709077 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a54c1-4669-4aca-8265-b957775f00f6 admin admin] PATH_INFO: `/v3/projects/878e4994ef914c2ba57e810d6e875a86` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:58.710189 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a54c1-4669-4aca-8265-b957775f00f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=878e4994ef914c2ba57e810d6e875a86)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:58.712151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a54c1-4669-4aca-8265-b957775f00f6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:58.760143 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e5a54c1-4669-4aca-8265-b957775f00f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '878e4994ef914c2ba57e810d6e875a86', 'request_id': 'req-8e5a54c1-4669-4aca-8265-b957775f00f6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:58.766778 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e5a54c1-4669-4aca-8265-b957775f00f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '878e4994ef914c2ba57e810d6e875a86', 'request_id': 'req-8e5a54c1-4669-4aca-8265-b957775f00f6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:58.779471 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e5a54c1-4669-4aca-8265-b957775f00f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '878e4994ef914c2ba57e810d6e875a86', 'request_id': 'req-8e5a54c1-4669-4aca-8265-b957775f00f6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:58.792473 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e5a54c1-4669-4aca-8265-b957775f00f6 admin admin] The token cache is being invalidate because project 878e4994ef914c2ba57e810d6e875a86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:23:58.792761 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e5a54c1-4669-4aca-8265-b957775f00f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:58.793377 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8e5a54c1-4669-4aca-8265-b957775f00f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:23:58.794865 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1430/2874] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:23:58 2022] DELETE /identity/v3/projects/878e4994ef914c2ba57e810d6e875a86 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:58.800337 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7924c-fbf1-4db2-86a4-f30174f24b2c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:58.800622 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7924c-fbf1-4db2-86a4-f30174f24b2c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:58.800912 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7924c-fbf1-4db2-86a4-f30174f24b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:58.820795 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ccc7924c-fbf1-4db2-86a4-f30174f24b2c None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:58.821995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ccc7924c-fbf1-4db2-86a4-f30174f24b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:58.852111 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:58.883738 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1445/2875] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:58.892352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb585c6-f5e8-4725-8f5d-2bc2a31dad6d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:58.892847 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb585c6-f5e8-4725-8f5d-2bc2a31dad6d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:58.893132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb585c6-f5e8-4725-8f5d-2bc2a31dad6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:58.912910 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5fb585c6-f5e8-4725-8f5d-2bc2a31dad6d None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:58.914117 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5fb585c6-f5e8-4725-8f5d-2bc2a31dad6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:58.921869 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1431/2876] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:58.926195 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7924c-fbf1-4db2-86a4-f30174f24b2c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:58.935894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7924c-fbf1-4db2-86a4-f30174f24b2c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:58.941116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46885bd6-f4ae-4610-99c4-290f5630425b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:58.941716 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46885bd6-f4ae-4610-99c4-290f5630425b admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:58.941995 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46885bd6-f4ae-4610-99c4-290f5630425b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:58.942268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46885bd6-f4ae-4610-99c4-290f5630425b admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:58.942905 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46885bd6-f4ae-4610-99c4-290f5630425b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:23:58.943251 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46885bd6-f4ae-4610-99c4-290f5630425b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:58.944843 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46885bd6-f4ae-4610-99c4-290f5630425b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:58.957588 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1446/2877] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:23:58.961956 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb585c6-f5e8-4725-8f5d-2bc2a31dad6d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:58.971222 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb585c6-f5e8-4725-8f5d-2bc2a31dad6d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:58.976017 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe1ae3-f991-4934-86a6-cfc244ec98b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:58.976609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe1ae3-f991-4934-86a6-cfc244ec98b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:58.976882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe1ae3-f991-4934-86a6-cfc244ec98b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:58.977151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe1ae3-f991-4934-86a6-cfc244ec98b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:58.977781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe1ae3-f991-4934-86a6-cfc244ec98b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:58.979750 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe1ae3-f991-4934-86a6-cfc244ec98b9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:59.007946 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1432/2878] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:23:58 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:23:59.012140 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46885bd6-f4ae-4610-99c4-290f5630425b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:59.022305 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46885bd6-f4ae-4610-99c4-290f5630425b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:59.079068 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883b3cf-f494-49f4-ab36-d987651fb7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:59.079679 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883b3cf-f494-49f4-ab36-d987651fb7b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:59.079960 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883b3cf-f494-49f4-ab36-d987651fb7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:59.080236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883b3cf-f494-49f4-ab36-d987651fb7b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:59.080857 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883b3cf-f494-49f4-ab36-d987651fb7b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:59.082771 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883b3cf-f494-49f4-ab36-d987651fb7b9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:59.104497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1447/2879] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:23:59 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:23:59.108633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe1ae3-f991-4934-86a6-cfc244ec98b9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:59.119559 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe1ae3-f991-4934-86a6-cfc244ec98b9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:59.124183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380d22c-9968-4ca0-a4c9-7a69a8dcd37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:59.124782 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380d22c-9968-4ca0-a4c9-7a69a8dcd37a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:59.125067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380d22c-9968-4ca0-a4c9-7a69a8dcd37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:59.126226 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380d22c-9968-4ca0-a4c9-7a69a8dcd37a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:59.126996 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380d22c-9968-4ca0-a4c9-7a69a8dcd37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:59.128596 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380d22c-9968-4ca0-a4c9-7a69a8dcd37a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:59.138360 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1433/2880] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:59.142831 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883b3cf-f494-49f4-ab36-d987651fb7b9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:59.152794 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883b3cf-f494-49f4-ab36-d987651fb7b9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:59.157503 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195d207-990e-479d-9af1-120597cecccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:59.158102 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195d207-990e-479d-9af1-120597cecccb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:59.158384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195d207-990e-479d-9af1-120597cecccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:59.158685 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195d207-990e-479d-9af1-120597cecccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:59.159395 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195d207-990e-479d-9af1-120597cecccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:59.160980 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195d207-990e-479d-9af1-120597cecccb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:59.171478 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1448/2881] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:23:59.176317 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380d22c-9968-4ca0-a4c9-7a69a8dcd37a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:59.186363 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380d22c-9968-4ca0-a4c9-7a69a8dcd37a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:59.191633 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963033a-f571-4497-a623-9b27958c6ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:59.192278 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963033a-f571-4497-a623-9b27958c6ced admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:59.192557 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963033a-f571-4497-a623-9b27958c6ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:59.192831 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963033a-f571-4497-a623-9b27958c6ced admin admin] PATH_INFO: `/v3/projects/658e8f7418634b188c28aae0981c02d8/users/77e2098718c04d80a99d4be1d6693e86/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:59.207238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963033a-f571-4497-a623-9b27958c6ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=658e8f7418634b188c28aae0981c02d8, user_id=77e2098718c04d80a99d4be1d6693e86, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:59.209890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963033a-f571-4497-a623-9b27958c6ced admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:59.223859 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1434/2882] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:23:59 2022] PUT /identity/v3/projects/658e8f7418634b188c28aae0981c02d8/users/77e2098718c04d80a99d4be1d6693e86/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:23:59.235809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a703f7-f1eb-4df9-a155-b7b3105ad84e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:59.236087 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a703f7-f1eb-4df9-a155-b7b3105ad84e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:59.236371 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a703f7-f1eb-4df9-a155-b7b3105ad84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:59.265680 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-77a703f7-f1eb-4df9-a155-b7b3105ad84e None None] MFA Rules not processed for user `77e2098718c04d80a99d4be1d6693e86`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:23:59.266992 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-77a703f7-f1eb-4df9-a155-b7b3105ad84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:59.329936 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1449/2883] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:23:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:23:59.340452 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963033a-f571-4497-a623-9b27958c6ced admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:23:59.342195 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9963033a-f571-4497-a623-9b27958c6ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:23:59.385009 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:23:59.401734 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963033a-f571-4497-a623-9b27958c6ced admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:23:59.405362 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d291b-03d7-4aeb-bcfe-bea244bfbc72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:23:59.405961 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d291b-03d7-4aeb-bcfe-bea244bfbc72 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:23:59.406235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d291b-03d7-4aeb-bcfe-bea244bfbc72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:23:59.406504 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d291b-03d7-4aeb-bcfe-bea244bfbc72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:23:59.415175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d291b-03d7-4aeb-bcfe-bea244bfbc72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:23:59.416610 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d291b-03d7-4aeb-bcfe-bea244bfbc72 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:23:59.426041 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1435/2884] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:23:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:24:00.777074 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c876b-9c96-4e57-b95f-f767ac8c622b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:00.777579 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c876b-9c96-4e57-b95f-f767ac8c622b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:00.777684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c876b-9c96-4e57-b95f-f767ac8c622b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:00.796351 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c25c876b-9c96-4e57-b95f-f767ac8c622b None None] MFA Rules not processed for user `77e2098718c04d80a99d4be1d6693e86`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:24:00.798209 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c25c876b-9c96-4e57-b95f-f767ac8c622b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:00.805297 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1450/2885] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:24:09.421343 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=9 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:24:10.790986 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:24:13.375414 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953be85-4127-4bef-ab1d-d14b0594c2cc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:13.376133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953be85-4127-4bef-ab1d-d14b0594c2cc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:13.376235 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953be85-4127-4bef-ab1d-d14b0594c2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:13.396062 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f953be85-4127-4bef-ab1d-d14b0594c2cc None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:24:13.397252 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f953be85-4127-4bef-ab1d-d14b0594c2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:13.450769 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1436/2886] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:24:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:24:13.496769 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c876b-9c96-4e57-b95f-f767ac8c622b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:13.498197 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c25c876b-9c96-4e57-b95f-f767ac8c622b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:13.572006 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c876b-9c96-4e57-b95f-f767ac8c622b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:13.577063 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99414a4-4026-4cd5-a9db-9511eb914a22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:13.577674 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99414a4-4026-4cd5-a9db-9511eb914a22 service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:13.577953 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99414a4-4026-4cd5-a9db-9511eb914a22 service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:13.578754 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99414a4-4026-4cd5-a9db-9511eb914a22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:13.587097 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99414a4-4026-4cd5-a9db-9511eb914a22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:13.588695 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99414a4-4026-4cd5-a9db-9511eb914a22 service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:13.600015 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1451/2887] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:24:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:24:14.123249 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144448c-cbb9-471e-84f3-1297ae40756a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:14.123534 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144448c-cbb9-471e-84f3-1297ae40756a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:14.123813 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144448c-cbb9-471e-84f3-1297ae40756a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:14.143546 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b144448c-cbb9-471e-84f3-1297ae40756a None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:24:14.144743 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b144448c-cbb9-471e-84f3-1297ae40756a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:14.151829 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1437/2888] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 9 13:24:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:24:14.164123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99414a4-4026-4cd5-a9db-9511eb914a22 service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:14.175204 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99414a4-4026-4cd5-a9db-9511eb914a22 service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:14.180593 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd869a8f-65a2-4606-a820-5a0b0a43ebdd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:14.181187 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd869a8f-65a2-4606-a820-5a0b0a43ebdd service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:14.181464 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd869a8f-65a2-4606-a820-5a0b0a43ebdd service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:14.181735 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd869a8f-65a2-4606-a820-5a0b0a43ebdd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:14.190418 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd869a8f-65a2-4606-a820-5a0b0a43ebdd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:14.192012 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd869a8f-65a2-4606-a820-5a0b0a43ebdd service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:14.200038 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1452/2889] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:24:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:24:14.424286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45fa6-132b-484e-afde-671c661d8e22 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:14.424774 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45fa6-132b-484e-afde-671c661d8e22 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:14.424872 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45fa6-132b-484e-afde-671c661d8e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:14.443677 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-75a45fa6-132b-484e-afde-671c661d8e22 None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:24:14.444868 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-75a45fa6-132b-484e-afde-671c661d8e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:14.452487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1438/2890] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 9 13:24:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:24:14.464492 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd869a8f-65a2-4606-a820-5a0b0a43ebdd service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:14.475611 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd869a8f-65a2-4606-a820-5a0b0a43ebdd service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:14.480693 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b9e23-99aa-4663-ac26-832a92101d18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:14.481278 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b9e23-99aa-4663-ac26-832a92101d18 service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:14.481553 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b9e23-99aa-4663-ac26-832a92101d18 service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:14.481828 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b9e23-99aa-4663-ac26-832a92101d18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:14.490381 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b9e23-99aa-4663-ac26-832a92101d18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:14.491977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b9e23-99aa-4663-ac26-832a92101d18 service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:14.500220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1453/2891] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:24:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 13:24:14.988399 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff28beb-d1e0-42d7-8ecd-0fa40d518d59 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:14.988686 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff28beb-d1e0-42d7-8ecd-0fa40d518d59 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:14.988959 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff28beb-d1e0-42d7-8ecd-0fa40d518d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:15.014958 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cff28beb-d1e0-42d7-8ecd-0fa40d518d59 None None] MFA Rules not processed for user `8d343a3db21c47e1a85a739a55559c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:24:15.016886 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cff28beb-d1e0-42d7-8ecd-0fa40d518d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:15.024119 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1439/2892] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 9 13:24:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:24:15.036445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b9e23-99aa-4663-ac26-832a92101d18 service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:15.046787 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b9e23-99aa-4663-ac26-832a92101d18 service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:15.052358 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde54b8-5e19-44a1-b2e1-d88ad4b65057 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c361a3e83f8e45c2b4d28f97f0a9d069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:15.052946 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde54b8-5e19-44a1-b2e1-d88ad4b65057 service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:15.053220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde54b8-5e19-44a1-b2e1-d88ad4b65057 service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:15.053495 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde54b8-5e19-44a1-b2e1-d88ad4b65057 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:15.061384 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde54b8-5e19-44a1-b2e1-d88ad4b65057 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:15.063765 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde54b8-5e19-44a1-b2e1-d88ad4b65057 service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:15.071964 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1454/2893] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 9 13:24:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:24:17.310366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff28beb-d1e0-42d7-8ecd-0fa40d518d59 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:17.372488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff28beb-d1e0-42d7-8ecd-0fa40d518d59 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:17.377807 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:17.378436 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:17.378715 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:17.378988 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] PATH_INFO: `/v3/users/77e2098718c04d80a99d4be1d6693e86` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:17.380150 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=77e2098718c04d80a99d4be1d6693e86)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:17.382067 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:17.426963 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77e2098718c04d80a99d4be1d6693e86'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:24:17.434272 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77e2098718c04d80a99d4be1d6693e86'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:24:17.446964 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77e2098718c04d80a99d4be1d6693e86'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:24:17.447631 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77e2098718c04d80a99d4be1d6693e86'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:24:17.455361 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77e2098718c04d80a99d4be1d6693e86'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:24:17.461195 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1440/2894] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:24:17 2022] DELETE /identity/v3/users/77e2098718c04d80a99d4be1d6693e86 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:24:17.465630 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5718415-915c-47ec-a887-db8f4f06ea78 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:17.466108 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5718415-915c-47ec-a887-db8f4f06ea78 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:17.466244 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5718415-915c-47ec-a887-db8f4f06ea78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:17.486462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b5718415-915c-47ec-a887-db8f4f06ea78 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:24:17.487759 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b5718415-915c-47ec-a887-db8f4f06ea78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:17.546790 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1455/2895] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:24:17.556366 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:17.557764 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:17.565044 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:24:17.619683 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b904fab-9afb-440c-8701-853ec893ad3a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:17.623389 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac562bd3-0ff5-4071-af9d-59f2068b0cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:17.623999 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac562bd3-0ff5-4071-af9d-59f2068b0cdd service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:17.624281 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac562bd3-0ff5-4071-af9d-59f2068b0cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:17.624556 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac562bd3-0ff5-4071-af9d-59f2068b0cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:17.634986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac562bd3-0ff5-4071-af9d-59f2068b0cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:17.636490 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac562bd3-0ff5-4071-af9d-59f2068b0cdd service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:17.646737 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1441/2896] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:24:17.815069 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5718415-915c-47ec-a887-db8f4f06ea78 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:17.816524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b5718415-915c-47ec-a887-db8f4f06ea78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:17.832595 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5718415-915c-47ec-a887-db8f4f06ea78 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:17.837436 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204aab6-67c4-4dda-9633-4b288d20611d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:17.838236 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204aab6-67c4-4dda-9633-4b288d20611d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:17.838635 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204aab6-67c4-4dda-9633-4b288d20611d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:17.839025 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204aab6-67c4-4dda-9633-4b288d20611d admin admin] PATH_INFO: `/v3/projects/658e8f7418634b188c28aae0981c02d8` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:17.840777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204aab6-67c4-4dda-9633-4b288d20611d admin admin] RBAC: Authorizing `identity:delete_project(project_id=658e8f7418634b188c28aae0981c02d8)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:17.842716 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204aab6-67c4-4dda-9633-4b288d20611d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:17.887883 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:24:17.892123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c204aab6-67c4-4dda-9633-4b288d20611d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658e8f7418634b188c28aae0981c02d8', 'request_id': 'req-c204aab6-67c4-4dda-9633-4b288d20611d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:24:17.899878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c204aab6-67c4-4dda-9633-4b288d20611d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658e8f7418634b188c28aae0981c02d8', 'request_id': 'req-c204aab6-67c4-4dda-9633-4b288d20611d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:24:17.905546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c204aab6-67c4-4dda-9633-4b288d20611d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658e8f7418634b188c28aae0981c02d8', 'request_id': 'req-c204aab6-67c4-4dda-9633-4b288d20611d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:24:17.917285 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c204aab6-67c4-4dda-9633-4b288d20611d admin admin] The token cache is being invalidate because project 658e8f7418634b188c28aae0981c02d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:24:17.917574 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c204aab6-67c4-4dda-9633-4b288d20611d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:24:17.918219 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c204aab6-67c4-4dda-9633-4b288d20611d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:24:17.919734 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1456/2897] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:24:17 2022] DELETE /identity/v3/projects/658e8f7418634b188c28aae0981c02d8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:24:17.924845 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b0c0b-88a9-448c-8ffc-cb0bbcfbc16f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:17.925133 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b0c0b-88a9-448c-8ffc-cb0bbcfbc16f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:17.925409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b0c0b-88a9-448c-8ffc-cb0bbcfbc16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:17.944525 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1a0b0c0b-88a9-448c-8ffc-cb0bbcfbc16f None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:24:17.945725 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1a0b0c0b-88a9-448c-8ffc-cb0bbcfbc16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:18.005008 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1442/2898] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:24:18.013574 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc13f9-71c1-4725-be0d-c8d78a99cebe None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:18.013865 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc13f9-71c1-4725-be0d-c8d78a99cebe None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:18.014143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc13f9-71c1-4725-be0d-c8d78a99cebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:18.035724 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dbfc13f9-71c1-4725-be0d-c8d78a99cebe None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:24:18.036931 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dbfc13f9-71c1-4725-be0d-c8d78a99cebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:18.044761 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1457/2899] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:24:18.048799 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b0c0b-88a9-448c-8ffc-cb0bbcfbc16f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:18.058270 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b0c0b-88a9-448c-8ffc-cb0bbcfbc16f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:18.062730 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4c868-a384-4cc1-9c5b-34e69a32d11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:18.063358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4c868-a384-4cc1-9c5b-34e69a32d11c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:18.063637 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4c868-a384-4cc1-9c5b-34e69a32d11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:18.063907 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4c868-a384-4cc1-9c5b-34e69a32d11c admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:18.064549 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4c868-a384-4cc1-9c5b-34e69a32d11c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:24:18.064852 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4c868-a384-4cc1-9c5b-34e69a32d11c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:18.066444 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4c868-a384-4cc1-9c5b-34e69a32d11c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:18.077746 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1443/2900] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 9 13:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:24:18.082542 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc13f9-71c1-4725-be0d-c8d78a99cebe None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:18.091809 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc13f9-71c1-4725-be0d-c8d78a99cebe None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:18.097343 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa4001-db59-4a78-ab05-fbc0414e6123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:18.097940 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa4001-db59-4a78-ab05-fbc0414e6123 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:18.098220 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa4001-db59-4a78-ab05-fbc0414e6123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:18.098494 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa4001-db59-4a78-ab05-fbc0414e6123 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:18.099213 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa4001-db59-4a78-ab05-fbc0414e6123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:18.101374 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa4001-db59-4a78-ab05-fbc0414e6123 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:18.128842 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1458/2901] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:24:18 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:24:18.133036 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4c868-a384-4cc1-9c5b-34e69a32d11c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:18.143470 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4c868-a384-4cc1-9c5b-34e69a32d11c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:18.197355 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd46ce9-5f06-4f42-a6a2-627f4823cceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:18.197953 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd46ce9-5f06-4f42-a6a2-627f4823cceb admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:18.198234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd46ce9-5f06-4f42-a6a2-627f4823cceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:18.198507 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd46ce9-5f06-4f42-a6a2-627f4823cceb admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:18.199164 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd46ce9-5f06-4f42-a6a2-627f4823cceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:18.201075 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd46ce9-5f06-4f42-a6a2-627f4823cceb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:18.221358 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1444/2902] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:24:18 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:24:18.225546 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa4001-db59-4a78-ab05-fbc0414e6123 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:18.235699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa4001-db59-4a78-ab05-fbc0414e6123 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:18.240224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1878a50-9f61-42d6-9379-6ab42577176c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:18.240817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1878a50-9f61-42d6-9379-6ab42577176c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:18.241091 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1878a50-9f61-42d6-9379-6ab42577176c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:18.241359 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1878a50-9f61-42d6-9379-6ab42577176c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:18.241990 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1878a50-9f61-42d6-9379-6ab42577176c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:18.243591 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1878a50-9f61-42d6-9379-6ab42577176c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:18.254129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1459/2903] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:24:18.258081 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd46ce9-5f06-4f42-a6a2-627f4823cceb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:18.268612 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd46ce9-5f06-4f42-a6a2-627f4823cceb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:18.273224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c8384-7b69-40da-9098-3268770049f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:18.273817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c8384-7b69-40da-9098-3268770049f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:18.274095 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c8384-7b69-40da-9098-3268770049f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:18.274365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c8384-7b69-40da-9098-3268770049f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:18.274997 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c8384-7b69-40da-9098-3268770049f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:18.276615 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c8384-7b69-40da-9098-3268770049f8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:18.288635 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1445/2904] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:24:18.292699 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1878a50-9f61-42d6-9379-6ab42577176c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:18.302667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1878a50-9f61-42d6-9379-6ab42577176c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:18.307322 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e7fd6-ee14-4fec-a857-3f1f100bec14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:18.307997 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e7fd6-ee14-4fec-a857-3f1f100bec14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:18.308329 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e7fd6-ee14-4fec-a857-3f1f100bec14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:18.308553 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e7fd6-ee14-4fec-a857-3f1f100bec14 admin admin] PATH_INFO: `/v3/projects/b7dd3b78c2a74ac4baa2f0bb21482ed9/users/5ab98652cbab470b8a385fe6381dd80b/roles/e7b95eb3b5ad4479b50aaa68e58591ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:18.323528 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e7fd6-ee14-4fec-a857-3f1f100bec14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7dd3b78c2a74ac4baa2f0bb21482ed9, user_id=5ab98652cbab470b8a385fe6381dd80b, role_id=e7b95eb3b5ad4479b50aaa68e58591ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:18.326176 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e7fd6-ee14-4fec-a857-3f1f100bec14 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:18.340027 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1460/2905] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:24:18 2022] PUT /identity/v3/projects/b7dd3b78c2a74ac4baa2f0bb21482ed9/users/5ab98652cbab470b8a385fe6381dd80b/roles/e7b95eb3b5ad4479b50aaa68e58591ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:24:18.345040 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb1ee4-66a1-40df-806d-ea408fbc7815 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:18.345501 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb1ee4-66a1-40df-806d-ea408fbc7815 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:18.345635 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb1ee4-66a1-40df-806d-ea408fbc7815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:18.364099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-faeb1ee4-66a1-40df-806d-ea408fbc7815 None None] MFA Rules not processed for user `a4f7da3ad6214b7b94cd5b55e583b917`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:24:18.365287 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-faeb1ee4-66a1-40df-806d-ea408fbc7815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:18.373099 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:24:18.424282 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1446/2906] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:24:18.433732 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e7fd6-ee14-4fec-a857-3f1f100bec14 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:18.435445 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-407e7fd6-ee14-4fec-a857-3f1f100bec14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:18.493061 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e7fd6-ee14-4fec-a857-3f1f100bec14 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:18.496800 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6d7d4-fb25-4bfe-93ee-c856f67342a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:18.497404 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6d7d4-fb25-4bfe-93ee-c856f67342a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:18.497683 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6d7d4-fb25-4bfe-93ee-c856f67342a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:18.497956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6d7d4-fb25-4bfe-93ee-c856f67342a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:18.507946 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6d7d4-fb25-4bfe-93ee-c856f67342a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:18.509369 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6d7d4-fb25-4bfe-93ee-c856f67342a7 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:18.519023 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1461/2907] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:24:22.508510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef0c14-4709-418b-93dc-133d9720f455 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:22.509041 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef0c14-4709-418b-93dc-133d9720f455 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:22.509041 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef0c14-4709-418b-93dc-133d9720f455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:22.540879 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-30ef0c14-4709-418b-93dc-133d9720f455 None None] MFA Rules not processed for user `5ab98652cbab470b8a385fe6381dd80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:24:22.542039 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-30ef0c14-4709-418b-93dc-133d9720f455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:22.610522 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1447/2908] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:24:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:24:22.615373 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6d7d4-fb25-4bfe-93ee-c856f67342a7 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:22.626463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6d7d4-fb25-4bfe-93ee-c856f67342a7 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:22.632073 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15441b01-2645-4013-aa59-3f6b3785a89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:22.632683 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15441b01-2645-4013-aa59-3f6b3785a89d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:22.632969 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15441b01-2645-4013-aa59-3f6b3785a89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:22.633241 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15441b01-2645-4013-aa59-3f6b3785a89d admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:22.633877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15441b01-2645-4013-aa59-3f6b3785a89d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:22.635855 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15441b01-2645-4013-aa59-3f6b3785a89d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:22.665106 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1462/2909] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 9 13:24:22 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:24:22.668893 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef0c14-4709-418b-93dc-133d9720f455 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:22.679983 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef0c14-4709-418b-93dc-133d9720f455 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:22.736798 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f2be7-61e1-4dcb-a495-abee266a837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:22.737402 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f2be7-61e1-4dcb-a495-abee266a837f admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:22.737678 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f2be7-61e1-4dcb-a495-abee266a837f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:22.737947 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f2be7-61e1-4dcb-a495-abee266a837f admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:22.738577 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f2be7-61e1-4dcb-a495-abee266a837f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:22.740516 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f2be7-61e1-4dcb-a495-abee266a837f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:22.762286 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1448/2910] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 9 13:24:22 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:24:22.766876 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15441b01-2645-4013-aa59-3f6b3785a89d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:22.777081 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15441b01-2645-4013-aa59-3f6b3785a89d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:22.781879 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea6a69-1b13-45cc-8ce0-09b0e4f78ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:22.782717 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea6a69-1b13-45cc-8ce0-09b0e4f78ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:22.783005 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea6a69-1b13-45cc-8ce0-09b0e4f78ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:22.783309 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea6a69-1b13-45cc-8ce0-09b0e4f78ac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:22.784143 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea6a69-1b13-45cc-8ce0-09b0e4f78ac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:22.785744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea6a69-1b13-45cc-8ce0-09b0e4f78ac7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:22.796844 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1463/2911] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:24:22.801118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f2be7-61e1-4dcb-a495-abee266a837f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:22.811699 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f2be7-61e1-4dcb-a495-abee266a837f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:22.816944 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd99a01-01bc-4940-a932-cd27bd3f2d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:22.817598 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd99a01-01bc-4940-a932-cd27bd3f2d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:22.817877 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd99a01-01bc-4940-a932-cd27bd3f2d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:22.818147 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd99a01-01bc-4940-a932-cd27bd3f2d7d admin admin] PATH_INFO: `/v3/projects/c0385385285b4e7dbc5a198d16214b85/users/46b172f71a42416d8d4856971182ff5b/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:22.834125 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd99a01-01bc-4940-a932-cd27bd3f2d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0385385285b4e7dbc5a198d16214b85, user_id=46b172f71a42416d8d4856971182ff5b, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:22.836817 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd99a01-01bc-4940-a932-cd27bd3f2d7d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:22.849544 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1449/2912] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:24:22 2022] PUT /identity/v3/projects/c0385385285b4e7dbc5a198d16214b85/users/46b172f71a42416d8d4856971182ff5b/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:24:22.853612 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea6a69-1b13-45cc-8ce0-09b0e4f78ac7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:22.863629 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea6a69-1b13-45cc-8ce0-09b0e4f78ac7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:22.902211 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:24:22.922205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632ab02-231b-4669-a579-04d4682fe067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:22.922814 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632ab02-231b-4669-a579-04d4682fe067 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:22.923229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632ab02-231b-4669-a579-04d4682fe067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:22.923522 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632ab02-231b-4669-a579-04d4682fe067 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:22.924158 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632ab02-231b-4669-a579-04d4682fe067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:22.925744 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632ab02-231b-4669-a579-04d4682fe067 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:22.935477 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1464/2913] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 9 13:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:24:22.940459 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd99a01-01bc-4940-a932-cd27bd3f2d7d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:22.951224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd99a01-01bc-4940-a932-cd27bd3f2d7d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:22.956891 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00284174-0d5b-411b-9a24-76acb771892b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:22.957548 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284174-0d5b-411b-9a24-76acb771892b admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:22.957827 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284174-0d5b-411b-9a24-76acb771892b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:22.958098 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284174-0d5b-411b-9a24-76acb771892b admin admin] PATH_INFO: `/v3/projects/c0385385285b4e7dbc5a198d16214b85/users/46b172f71a42416d8d4856971182ff5b/roles/8e1f1c5e55d24237b79880bb43f41ee6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:22.960056 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00284174-0d5b-411b-9a24-76acb771892b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0385385285b4e7dbc5a198d16214b85, user_id=46b172f71a42416d8d4856971182ff5b, role_id=8e1f1c5e55d24237b79880bb43f41ee6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:22.962691 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00284174-0d5b-411b-9a24-76acb771892b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:22.965871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:24:22.974789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1450/2914] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 9 13:24:22 2022] PUT /identity/v3/projects/c0385385285b4e7dbc5a198d16214b85/users/46b172f71a42416d8d4856971182ff5b/roles/8e1f1c5e55d24237b79880bb43f41ee6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:24:27.104123 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d7b2e-bdd9-4b44-8369-0b4aeabdbc12 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:27.104648 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d7b2e-bdd9-4b44-8369-0b4aeabdbc12 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:27.104955 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d7b2e-bdd9-4b44-8369-0b4aeabdbc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:27.141714 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2e0d7b2e-bdd9-4b44-8369-0b4aeabdbc12 None None] MFA Rules not processed for user `46b172f71a42416d8d4856971182ff5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:24:27.143812 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2e0d7b2e-bdd9-4b44-8369-0b4aeabdbc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:27.267473 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1465/2915] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:24:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:24:27.288912 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00284174-0d5b-411b-9a24-76acb771892b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:27.290815 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-00284174-0d5b-411b-9a24-76acb771892b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:27.413609 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00284174-0d5b-411b-9a24-76acb771892b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:27.423600 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3f176-906e-4cdc-96d7-792c941a04cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:27.424237 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3f176-906e-4cdc-96d7-792c941a04cc service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:27.424513 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3f176-906e-4cdc-96d7-792c941a04cc service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:27.424784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3f176-906e-4cdc-96d7-792c941a04cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:27.493733 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3f176-906e-4cdc-96d7-792c941a04cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:27.495365 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3f176-906e-4cdc-96d7-792c941a04cc service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:27.507316 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1451/2916] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:24:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:24:37.261841 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=45 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:24:37.502352 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=67 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:24:42.792490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bf6a3-07f9-4078-89c3-c832fb8db57c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:42.793037 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bf6a3-07f9-4078-89c3-c832fb8db57c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:42.793037 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bf6a3-07f9-4078-89c3-c832fb8db57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:42.816913 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c85bf6a3-07f9-4078-89c3-c832fb8db57c None None] MFA Rules not processed for user `5ab98652cbab470b8a385fe6381dd80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:24:42.819072 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c85bf6a3-07f9-4078-89c3-c832fb8db57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:42.827205 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1466/2917] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:24:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:24:42.869264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3f176-906e-4cdc-96d7-792c941a04cc service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:42.871309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b8d3f176-906e-4cdc-96d7-792c941a04cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:42.931911 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3f176-906e-4cdc-96d7-792c941a04cc service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:42.935495 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c303ef5-ac53-4776-8186-15ff4ca49446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea77fa8f5b754a2287f0639717f6e57a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:42.936132 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c303ef5-ac53-4776-8186-15ff4ca49446 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:42.936409 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c303ef5-ac53-4776-8186-15ff4ca49446 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:42.936679 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c303ef5-ac53-4776-8186-15ff4ca49446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:42.939035 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4c303ef5-ac53-4776-8186-15ff4ca49446 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:42.952603 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c303ef5-ac53-4776-8186-15ff4ca49446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:42.954161 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c303ef5-ac53-4776-8186-15ff4ca49446 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:42.965175 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1452/2918] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:24:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:24:43.737490 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bf6a3-07f9-4078-89c3-c832fb8db57c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:43.738938 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c85bf6a3-07f9-4078-89c3-c832fb8db57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:43.805334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bf6a3-07f9-4078-89c3-c832fb8db57c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:43.809598 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd871cdc-9387-4757-a40b-22d233212d68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:43.810224 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd871cdc-9387-4757-a40b-22d233212d68 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:43.810513 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd871cdc-9387-4757-a40b-22d233212d68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:43.810799 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd871cdc-9387-4757-a40b-22d233212d68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:43.813514 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fd871cdc-9387-4757-a40b-22d233212d68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:24:43.829664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd871cdc-9387-4757-a40b-22d233212d68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:43.831180 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd871cdc-9387-4757-a40b-22d233212d68 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:43.842959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1467/2919] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:24:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:24:52.960244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=26 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:24:53.837692 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=29 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:24:59.414855 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c303ef5-ac53-4776-8186-15ff4ca49446 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:24:59.424840 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c303ef5-ac53-4776-8186-15ff4ca49446 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:24:59.428440 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607786d-ad35-4364-8647-07242d274cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:24:59.429033 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607786d-ad35-4364-8647-07242d274cad service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:24:59.429308 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607786d-ad35-4364-8647-07242d274cad service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:24:59.429579 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607786d-ad35-4364-8647-07242d274cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:24:59.438309 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607786d-ad35-4364-8647-07242d274cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:24:59.439771 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607786d-ad35-4364-8647-07242d274cad service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:24:59.449078 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1453/2920] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:24:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:25:09.444957 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:25:53.663433 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd871cdc-9387-4757-a40b-22d233212d68 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:25:53.672323 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd871cdc-9387-4757-a40b-22d233212d68 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:25:53.676777 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71784200-b959-4c9a-9904-0128afdfb2be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:25:53.677380 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71784200-b959-4c9a-9904-0128afdfb2be service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:25:53.677662 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71784200-b959-4c9a-9904-0128afdfb2be service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:25:53.677934 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71784200-b959-4c9a-9904-0128afdfb2be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:25:53.679660 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-71784200-b959-4c9a-9904-0128afdfb2be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:25:53.689295 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71784200-b959-4c9a-9904-0128afdfb2be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:25:53.690753 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71784200-b959-4c9a-9904-0128afdfb2be service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:25:53.698906 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1468/2921] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:25:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:26:03.695105 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:26:58.173725 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607786d-ad35-4364-8647-07242d274cad service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:26:58.195224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607786d-ad35-4364-8647-07242d274cad service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:26:58.203705 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1c56c-7529-49d8-b802-d1a3122c2a96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:26:58.204408 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1c56c-7529-49d8-b802-d1a3122c2a96 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:26:58.204880 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1c56c-7529-49d8-b802-d1a3122c2a96 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:26:58.205357 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1c56c-7529-49d8-b802-d1a3122c2a96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:26:58.207241 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-26b1c56c-7529-49d8-b802-d1a3122c2a96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:26:58.224507 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1c56c-7529-49d8-b802-d1a3122c2a96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:26:58.226301 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1c56c-7529-49d8-b802-d1a3122c2a96 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:26:58.243853 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1454/2922] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:26:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:26:59.176763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7520e-799e-460b-8cfb-b56b561e077f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:26:59.177235 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7520e-799e-460b-8cfb-b56b561e077f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:26:59.177355 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7520e-799e-460b-8cfb-b56b561e077f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:26:59.196638 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-37b7520e-799e-460b-8cfb-b56b561e077f None None] MFA Rules not processed for user `5ab98652cbab470b8a385fe6381dd80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:26:59.197823 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-37b7520e-799e-460b-8cfb-b56b561e077f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:26:59.204645 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1469/2923] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:26:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:26:59.953244 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1c56c-7529-49d8-b802-d1a3122c2a96 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:26:59.954748 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-26b1c56c-7529-49d8-b802-d1a3122c2a96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:27:00.027312 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1c56c-7529-49d8-b802-d1a3122c2a96 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:27:00.033488 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4005f-960d-4b7f-90a0-7536c127760c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:27:00.034094 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4005f-960d-4b7f-90a0-7536c127760c service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:27:00.034369 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4005f-960d-4b7f-90a0-7536c127760c service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:27:00.034634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4005f-960d-4b7f-90a0-7536c127760c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:27:00.036346 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4cd4005f-960d-4b7f-90a0-7536c127760c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:27:00.048338 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4005f-960d-4b7f-90a0-7536c127760c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:27:00.049905 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4005f-960d-4b7f-90a0-7536c127760c service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:27:00.060663 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1455/2924] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:26:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:27:09.191279 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:27:10.055486 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=38 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:27:13.362352 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7520e-799e-460b-8cfb-b56b561e077f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:27:13.375858 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7520e-799e-460b-8cfb-b56b561e077f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:27:13.381614 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc377901-97df-4717-9bb9-88f7e5d7af62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:27:13.382229 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc377901-97df-4717-9bb9-88f7e5d7af62 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:27:13.382510 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc377901-97df-4717-9bb9-88f7e5d7af62 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:27:13.382781 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc377901-97df-4717-9bb9-88f7e5d7af62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:27:13.384578 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fc377901-97df-4717-9bb9-88f7e5d7af62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:27:13.399507 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc377901-97df-4717-9bb9-88f7e5d7af62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:27:13.401109 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc377901-97df-4717-9bb9-88f7e5d7af62 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:27:13.412750 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1470/2925] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:27:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:27:23.408129 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:28:59.394510 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4005f-960d-4b7f-90a0-7536c127760c service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:28:59.396289 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4cd4005f-960d-4b7f-90a0-7536c127760c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:28:59.409726 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4005f-960d-4b7f-90a0-7536c127760c service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:28:59.414850 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409f9c9-68c1-4487-961e-01b7e51e2511 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:28:59.415511 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409f9c9-68c1-4487-961e-01b7e51e2511 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:28:59.415794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409f9c9-68c1-4487-961e-01b7e51e2511 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:28:59.416069 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409f9c9-68c1-4487-961e-01b7e51e2511 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:28:59.417871 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d409f9c9-68c1-4487-961e-01b7e51e2511 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:28:59.429295 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409f9c9-68c1-4487-961e-01b7e51e2511 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:28:59.430878 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409f9c9-68c1-4487-961e-01b7e51e2511 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:28:59.440896 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1456/2926] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:28:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:29:02.950972 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88279aaf-aa71-4207-b813-661ff942f392 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:29:02.951515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88279aaf-aa71-4207-b813-661ff942f392 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:29:02.951642 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88279aaf-aa71-4207-b813-661ff942f392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:29:02.970668 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-88279aaf-aa71-4207-b813-661ff942f392 None None] MFA Rules not processed for user `5ab98652cbab470b8a385fe6381dd80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:29:02.971902 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-88279aaf-aa71-4207-b813-661ff942f392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:29:02.978769 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1471/2927] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 9 13:29:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:29:03.304789 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409f9c9-68c1-4487-961e-01b7e51e2511 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:29:03.314230 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409f9c9-68c1-4487-961e-01b7e51e2511 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:29:03.317986 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83200b55-eb6d-4e2b-bc33-0620b688a34c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:29:03.318578 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83200b55-eb6d-4e2b-bc33-0620b688a34c service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:29:03.319690 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83200b55-eb6d-4e2b-bc33-0620b688a34c service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:29:03.319976 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83200b55-eb6d-4e2b-bc33-0620b688a34c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:29:03.321722 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-83200b55-eb6d-4e2b-bc33-0620b688a34c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:29:03.331197 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83200b55-eb6d-4e2b-bc33-0620b688a34c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:29:03.332620 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83200b55-eb6d-4e2b-bc33-0620b688a34c service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:29:03.340784 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1457/2928] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:29:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:29:12.965766 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:29:13.337176 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=12 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:30:56.333264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88279aaf-aa71-4207-b813-661ff942f392 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:30:56.343691 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88279aaf-aa71-4207-b813-661ff942f392 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:30:56.347221 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf94ea-f54f-42d0-a44a-a96de99d61ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:30:56.347817 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf94ea-f54f-42d0-a44a-a96de99d61ad service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:30:56.348141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf94ea-f54f-42d0-a44a-a96de99d61ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:30:56.348356 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf94ea-f54f-42d0-a44a-a96de99d61ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:30:56.356321 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf94ea-f54f-42d0-a44a-a96de99d61ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:30:56.358684 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf94ea-f54f-42d0-a44a-a96de99d61ad service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:30:56.367264 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1472/2929] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:30:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:30:57.280507 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83200b55-eb6d-4e2b-bc33-0620b688a34c service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:30:57.289518 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83200b55-eb6d-4e2b-bc33-0620b688a34c service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:30:57.294071 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee466d3-8f4a-44aa-bdaf-a455baabe68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:30:57.295564 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee466d3-8f4a-44aa-bdaf-a455baabe68c service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:30:57.295926 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee466d3-8f4a-44aa-bdaf-a455baabe68c service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:30:57.296238 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee466d3-8f4a-44aa-bdaf-a455baabe68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:30:57.305433 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee466d3-8f4a-44aa-bdaf-a455baabe68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:30:57.307111 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee466d3-8f4a-44aa-bdaf-a455baabe68c service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:30:57.318416 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1458/2930] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:30:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:30:57.408894 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf94ea-f54f-42d0-a44a-a96de99d61ad service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:30:57.417813 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf94ea-f54f-42d0-a44a-a96de99d61ad service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:30:57.421626 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24db8b-6e32-4efd-a440-4295b13720ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:30:57.422304 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24db8b-6e32-4efd-a440-4295b13720ff service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:30:57.422590 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24db8b-6e32-4efd-a440-4295b13720ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:30:57.422862 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24db8b-6e32-4efd-a440-4295b13720ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:30:57.433728 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24db8b-6e32-4efd-a440-4295b13720ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:30:57.435949 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24db8b-6e32-4efd-a440-4295b13720ff service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:30:57.447388 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1473/2931] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:30:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:30:58.634002 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee466d3-8f4a-44aa-bdaf-a455baabe68c service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:30:58.645224 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee466d3-8f4a-44aa-bdaf-a455baabe68c service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:30:58.649794 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9ab7e-e660-40a9-b3da-2845f1774135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:30:58.650430 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9ab7e-e660-40a9-b3da-2845f1774135 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:30:58.650709 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9ab7e-e660-40a9-b3da-2845f1774135 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:30:58.650979 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9ab7e-e660-40a9-b3da-2845f1774135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:30:58.665870 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9ab7e-e660-40a9-b3da-2845f1774135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:30:58.667410 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9ab7e-e660-40a9-b3da-2845f1774135 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:30:58.679882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1459/2932] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:30:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:31:04.042859 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24db8b-6e32-4efd-a440-4295b13720ff service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:31:04.055004 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24db8b-6e32-4efd-a440-4295b13720ff service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:31:04.061207 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76d67f-8c33-4903-b5b5-52ebf839c163 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441575b8b496457b905e9d5f5846e092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:31:04.061926 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d67f-8c33-4903-b5b5-52ebf839c163 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:31:04.062444 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d67f-8c33-4903-b5b5-52ebf839c163 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:31:04.062956 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d67f-8c33-4903-b5b5-52ebf839c163 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:31:04.064996 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6b76d67f-8c33-4903-b5b5-52ebf839c163 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:31:04.079954 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76d67f-8c33-4903-b5b5-52ebf839c163 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:31:04.081730 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76d67f-8c33-4903-b5b5-52ebf839c163 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:31:04.094917 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1474/2933] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:31:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:31:08.674462 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=12 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:31:09.727617 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9ab7e-e660-40a9-b3da-2845f1774135 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:31:09.737136 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9ab7e-e660-40a9-b3da-2845f1774135 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:31:09.740745 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099f4f4-3df8-4d9f-9067-1ec25dd51e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12deabe077b4095bfa7e2e8fac61d14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:31:09.741338 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099f4f4-3df8-4d9f-9067-1ec25dd51e33 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:31:09.741618 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099f4f4-3df8-4d9f-9067-1ec25dd51e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:31:09.741890 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099f4f4-3df8-4d9f-9067-1ec25dd51e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:31:09.802864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099f4f4-3df8-4d9f-9067-1ec25dd51e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:31:09.804264 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099f4f4-3df8-4d9f-9067-1ec25dd51e33 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:31:09.814232 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1460/2934] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:31:12.875993 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76d67f-8c33-4903-b5b5-52ebf839c163 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:31:12.891667 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76d67f-8c33-4903-b5b5-52ebf839c163 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:31:12.897460 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25a705-c2fd-4ede-95a0-919790eb06e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:31:12.898334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25a705-c2fd-4ede-95a0-919790eb06e0 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:31:12.898763 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25a705-c2fd-4ede-95a0-919790eb06e0 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:31:12.899071 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25a705-c2fd-4ede-95a0-919790eb06e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:31:12.900877 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cf25a705-c2fd-4ede-95a0-919790eb06e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:31:12.914421 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25a705-c2fd-4ede-95a0-919790eb06e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:31:12.916432 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25a705-c2fd-4ede-95a0-919790eb06e0 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:31:12.934065 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1475/2935] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:31:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:31:16.003858 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099f4f4-3df8-4d9f-9067-1ec25dd51e33 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:31:16.015657 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099f4f4-3df8-4d9f-9067-1ec25dd51e33 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:31:16.020860 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f5e3c-c561-48d1-82c9-eb62c5676683 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:31:16.021547 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f5e3c-c561-48d1-82c9-eb62c5676683 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:31:16.021882 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f5e3c-c561-48d1-82c9-eb62c5676683 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:31:16.022216 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f5e3c-c561-48d1-82c9-eb62c5676683 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:31:16.024047 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-121f5e3c-c561-48d1-82c9-eb62c5676683 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:31:16.036739 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f5e3c-c561-48d1-82c9-eb62c5676683 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:31:16.038331 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f5e3c-c561-48d1-82c9-eb62c5676683 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:31:16.049955 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1461/2936] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:31:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:31:16.066016 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25a705-c2fd-4ede-95a0-919790eb06e0 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:31:16.077675 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25a705-c2fd-4ede-95a0-919790eb06e0 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:31:16.083407 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:31:16.084227 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:31:16.086379 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:31:16.086960 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] PATH_INFO: `/v3/users/5ab98652cbab470b8a385fe6381dd80b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:31:16.088548 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ab98652cbab470b8a385fe6381dd80b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:31:16.090518 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:31:16.143977 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ab98652cbab470b8a385fe6381dd80b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.151933 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ab98652cbab470b8a385fe6381dd80b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.152619 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ab98652cbab470b8a385fe6381dd80b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.165914 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ab98652cbab470b8a385fe6381dd80b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.179116 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ab98652cbab470b8a385fe6381dd80b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.185377 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1476/2937] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:31:16 2022] DELETE /identity/v3/users/5ab98652cbab470b8a385fe6381dd80b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:31:16.361731 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f5e3c-c561-48d1-82c9-eb62c5676683 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:31:16.378151 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f5e3c-c561-48d1-82c9-eb62c5676683 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:31:16.434092 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:31:16.434718 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:31:16.434992 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:31:16.435313 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] PATH_INFO: `/v3/projects/b7dd3b78c2a74ac4baa2f0bb21482ed9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:31:16.436588 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7dd3b78c2a74ac4baa2f0bb21482ed9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:31:16.438903 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:31:16.485159 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7dd3b78c2a74ac4baa2f0bb21482ed9', 'request_id': 'req-e700f180-7992-4481-bce9-d4f504e8dfc6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.490479 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7dd3b78c2a74ac4baa2f0bb21482ed9', 'request_id': 'req-e700f180-7992-4481-bce9-d4f504e8dfc6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.496634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7dd3b78c2a74ac4baa2f0bb21482ed9', 'request_id': 'req-e700f180-7992-4481-bce9-d4f504e8dfc6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.508863 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] The token cache is being invalidate because project b7dd3b78c2a74ac4baa2f0bb21482ed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:31:16.509183 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.509824 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.511613 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1462/2938] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:31:16 2022] DELETE /identity/v3/projects/b7dd3b78c2a74ac4baa2f0bb21482ed9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:31:16.516268 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:31:16.517655 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:31:16.530921 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b5f5e-8c5c-4ad1-9f03-b9d846fb6bfe admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:31:16.588141 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:31:16.588773 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:31:16.589056 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:31:16.589334 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] PATH_INFO: `/v3/users/46b172f71a42416d8d4856971182ff5b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:31:16.590497 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46b172f71a42416d8d4856971182ff5b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:31:16.592462 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:31:16.614965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:31:16.641597 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46b172f71a42416d8d4856971182ff5b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.648959 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46b172f71a42416d8d4856971182ff5b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.649664 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46b172f71a42416d8d4856971182ff5b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.661999 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46b172f71a42416d8d4856971182ff5b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.668965 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46b172f71a42416d8d4856971182ff5b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.674878 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1477/2939] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 9 13:31:16 2022] DELETE /identity/v3/users/46b172f71a42416d8d4856971182ff5b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:31:16.847667 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:31:16.849114 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:31:16.862152 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700f180-7992-4481-bce9-d4f504e8dfc6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:31:16.870909 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:31:16.923315 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775065f4-e350-4e50-941c-e223157da52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f7da3ad6214b7b94cd5b55e583b917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97408104e346493ebafc4d549d2257c4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:31:16.923935 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775065f4-e350-4e50-941c-e223157da52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:31:16.924218 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775065f4-e350-4e50-941c-e223157da52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:31:16.924487 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775065f4-e350-4e50-941c-e223157da52e admin admin] PATH_INFO: `/v3/projects/c0385385285b4e7dbc5a198d16214b85` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:31:16.925634 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775065f4-e350-4e50-941c-e223157da52e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0385385285b4e7dbc5a198d16214b85)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:31:16.927604 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775065f4-e350-4e50-941c-e223157da52e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:31:16.975324 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-775065f4-e350-4e50-941c-e223157da52e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0385385285b4e7dbc5a198d16214b85', 'request_id': 'req-775065f4-e350-4e50-941c-e223157da52e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.980767 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-775065f4-e350-4e50-941c-e223157da52e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0385385285b4e7dbc5a198d16214b85', 'request_id': 'req-775065f4-e350-4e50-941c-e223157da52e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.987118 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-775065f4-e350-4e50-941c-e223157da52e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0385385285b4e7dbc5a198d16214b85', 'request_id': 'req-775065f4-e350-4e50-941c-e223157da52e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.998781 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-775065f4-e350-4e50-941c-e223157da52e admin admin] The token cache is being invalidate because project c0385385285b4e7dbc5a198d16214b85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:31:16.999234 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-775065f4-e350-4e50-941c-e223157da52e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:16.999900 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-775065f4-e350-4e50-941c-e223157da52e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:31:17.001676 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1463/2940] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 9 13:31:16 2022] DELETE /identity/v3/projects/c0385385285b4e7dbc5a198d16214b85 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:31:26.671463 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:31:26.995864 dv-cinder-831247-26-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:34:02.885524 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:34:02.887515 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:34:03.028952 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b567f41-8f90-4686-b03d-f9b70ea21b87 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:34:03.088202 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7c07e-25a9-4bbd-9c6d-dccf141feb9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbc7cef7b714463697dc1ee507e60be1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b842ac074b44e148c15882a7964aa13', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:34:03.088815 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7c07e-25a9-4bbd-9c6d-dccf141feb9a service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:34:03.089098 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7c07e-25a9-4bbd-9c6d-dccf141feb9a service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:34:03.089949 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7c07e-25a9-4bbd-9c6d-dccf141feb9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:34:03.091709 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-19b7c07e-25a9-4bbd-9c6d-dccf141feb9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:34:03.102722 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7c07e-25a9-4bbd-9c6d-dccf141feb9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:34:03.104342 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7c07e-25a9-4bbd-9c6d-dccf141feb9a service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:34:03.114326 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1478/2941] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 9 13:34:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:34:13.110005 dv-cinder-831247-26-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}